Development and validation of natural user interfaces for semantic enrichment of BIM models using open formats

Building information modelling (BIM) plays a prominent role among the technologies involved in Industry 4.0 (e.g. Internet of Things, mobile computing, virtual reality [VR], augmented reality [AR], big data, cloud computing, among many others) (Bakhshi et al., 2022; Maskuriy et al., 2019; Perrier et al., 2020). While some authors define it as the fourth industrial revolution (Newman et al., 2020; Perrier et al., 2020) combining information technology (IT), production and the internet (Pasetti Monizza et al., 2018) to automate traditional industry practices (Elghaish et al., 2021), there is still no commonly accepted definition for Industry 4.0 (Buer et al., 2018; Hofmann and Rüsch, 2017). An analogous concept is manifested in the understanding of Construction 4.0 (Oesterreich and Teuteberg, 2016), i.e. applications of Industry 4.0 to the construction sector (Elghaish et al., 2021). Construction 4.0 comprises the implementation of digital technologies throughout the construction sector (Karmakar and Delhi, 2021) to a greater level based on BIM systems coupling real and virtual building representations (Craveiro et al., 2019). Additionally, Construction 4.0 considers increased collaboration possibilities for stakeholders (Craveiro et al., 2019), although it requires a higher leap towards a paradigm shift in a sector recognised for its resistance to change (Arayici and Coates, 2012; Newman et al., 2020).

Despite initiatives carried out by public authorities and government bodies to foster the implementation and specify requirements for BIM use in public procurement (Ghaffarianhoseini et al., 2017; Smith, 2014), the BIM adoption rate has been slower than initially anticipated (Alreshidi et al., 2017; Walasek and Barszcz, 2017). Additionally, the need for a significant investment in training, education and additional software and hardware requirements (Ghaffarianhoseini et al., 2017; Liu et al., 2015; Plesner and Horst, 2013), followed by the reported lack of confidence, motivation, know-how and difference in skills towards BIM (Alreshidi et al., 2017; Walasek and Barszcz, 2017), present current challenges for a comprehensive uptake of BIM-based tools.

There is also a recognised need for increasing supportive technologies that enable the use of BIM, considering the available human resources, their requirements and the tasks that they perform (Kerosuo et al., 2015). In this regard, the development of immersive interfaces for the architecture, engineering, construction and operations sector (AECO) sector has provided new opportunities for collaboration (Boton, 2018; Cárcamo et al., 2017; Dinis and Poças Martins, 2016; Du et al., 2017, 2018; Lin et al., 2018) alongside further benefits such as reducing the technological skill gap by coupling natural user interfaces (NUIs) and head-mounted displays (HMD) (Dinis et al., 2020; Dinis and Poças Martins, 2016; Pour Rahimian et al., 2019). However, despite positive results reported by previous initiatives, a holistic, systematic assessment methodology or guidelines remains lacking (Sidani et al., 2021) to support the suitability of such interfaces to the AECO sector.

This study reflects on the relevance of developing innovative interfaces more attuned to the tasks, requirements and working environments considering the reported challenges to a full-fledged acceptance of the BIM methodology and the generalised use of BIM-based tools. Furthermore, this article intends to contribute to the body of knowledge by assessing the impact of NUIs and immersive interfaces on access to BIM information. Therefore, two research questions arise:

To answer the research questions, a framework is presented encompassing an open data transfer and storage system based on VR for the semantic enrichment of BIM models. Additionally, a case study was conducted following a thorough usability assessment methodology comprising formative and summative evaluations to assess the suitability of the proposed solution.

The following sections are organised as follows: Section 2 is dedicated to background research, literature review and related work; Section 3 describes the research approach and methodology of the study; Section 4 considers the system architecture overview and description of the proposed solution; the usability assessment and results are presented in Section 5; Section 6 includes the discussion of findings, and the article finishes with the conclusions presented in Section 7.

The application of IT equipment providing immersive visualisation and interaction with BIM models has become increasingly common in recent years (Sidani et al., 2021). In particular, technological developments (e.g. graphics and tracking technologies) (Nandavar et al., 2018; Wolfartsberger, 2019) and the increased affordability of HMDs (Hilfert and König, 2016) have enabled their use in other areas of knowledge beyond HMDs’ most common usage (i.e. the gaming industry) (Syamimi et al., 2020). In the AECO sector, the use of head-based VR platforms (Zhang et al., 2020) to devise BIM-based environments has enabled a substantial amount of research with favourable results in applications for collaboration within project teams (Boton, 2018; Du et al., 2017), facility management (Dinis et al., 2020; Yangming Shi et al., 2016), design review and supporting decision-making process (Dinis, 2016; Paes et al., 2021), construction safety (Azhar, 2017), engineering education and training (Dinis et al., 2017; Fogarty et al., 2018), among many others.

As collaboration and shared understanding are essential aspects when working with teams of different backgrounds and knowledge levels, the application of immersive interfaces (e.g. HMDs-based interfaces) may enable improved simulation of users’ 3D perception compared with non-immersive interfaces, thus enhancing spatial understanding (Paes et al., 2021).

Wolfartsberger (2019) describes the benefits of conducting design reviews through VR interfaces. Also, the author argues that in addition to the possible loss of the sense of scale when performing a design review through a screen (i.e. non-immersive interface), there is also the risk of excluding particular professional groups who are not entirely familiar with the type of software being used (Wolfartsberger, 2019).

As stated by Pour Rahimian et al. (2019), enhanced visualisation enabled by technologies such as VR promotes more democratic access to BIM models and improved understanding by non-technical professionals, therefore, overcoming the termed “black-box effect”.

Besides displaying BIM geometry, immersive environments are also used to access information concerning the project and its building elements, i.e. non-geometric information. Zhang et al. (2020) suggest that VR systems should display and retrieve specific project information capitalising on human interaction and attention, thus requiring further research effort in this field.

Commercial solutions have also been developed for this purpose, presenting alternative workflows to overcome interoperability hurdles between BIM authoring tools and other software, such as game engines, enabling the rapid development of immersive experiences. Unity Reflect Develop (Unity Reflect Develop, 2022) and Datasmith export plugins (Datasmith export plugins, 2022) are proprietary solutions to upload and link BIM models to VR scenes within a game engine.

Previous research initiatives have tackled open-source workflows to integrate non-geometrical information into immersive environments, extending the role of VR aside from the sheer visualisation of BIM models (Nandavar et al., 2018).

In that regard, Nandavar et al. (2018) devised a bidirectional data transfer solution based on industry foundation classes (IFC) for collaboration and layout safety planning. In detail, the authors highlight that exporting FBX (.fbx) or OBJ (.obj) files from BIM tools to Unity game entails several limitations, such as being a repetitive process and including loss of non-geometrical information (Nandavar et al., 2018). The solution proposed by Nandavar et al. (2018) comprises two layers. One layer is responsible for parsing the geometrical and metadata of the BIM model contained in a customised XML file and importing it to the game engine (i.e. Unity); a second layer concerning the conversion of changes made in VR to an XML file, afterwards parsed using the xBIM C-sharp (C#) toolkit and converted into a new IFC file. Also, the proposed VR prototype features a walkthrough, measurement, visualisation of building elements’ metadata (i.e. non-geometrical data of the BIM model), moving and deleting building elements, making points of interest and taking snapshots (Nandavar et al., 2018).

Hilfert and König (2016) describe a workflow to import BIM models’ geometry and material data as an IFC file into a game engine (i.e. unreal engine). The solution comprises a custom plugin to connect users to BIMServer, and then process the geometry as a binary representation, which is then parsed and displayed in the game engine. A custom database was also devised to map different materials to the correct building objects.

Pour Rahimian et al. (2019) present an IFC-based system to enhance stakeholder participation and collaboration in the design process, establishing real-time integration of BIM models into immersive environments. The proposed solution focuses on an openBIM cloud-centric approach and describes the development of a C# library to overcome compatibility issues between IFC and Unity’s geometry interpretation and enable developers to query and manipulate IFC entities. Also, a virtual showroom prototype to support client participation in the design process through an immersive VR environment is presented, featuring a model walkthrough, wall material and colour picker, an option to toggle light switches and the display and manipulation of IFC data, albeit little information is provided concerning user interaction with BIM information. The authors report that usability tests were conducted to allow further prototype iterations by gathering feedback from 20 participants. However, no information is provided regarding questions or the validity of the applied questionnaire.

Khalili (2021) presents a prototype solution to transfer geometric and semantic information from a BIM authoring tool to a game engine at runtime. The solution consists of a prototype that exchanges Autodesk Revit and Navisworks data to an XML file (forward process) through add-ins and applies changes made in VR into a new IFC file (backward process). Task schedules and clash detection information are provided within the VR environment dedicated to construction management information. Despite laboratory performance tests verifying the efficiency and possible benefits of the proposed solution, no user assessment tests were performed to verify the suitability of the system against AECO users’ requirements and needs.

Overall, data exchange between BIM and VR is still a demanding process faced with various interoperability hurdles despite reported research efforts (Khalili, 2021). Among the most commonly reported issues stands the data structure not being compatible from one software solution to the other (Khalili, 2021; Potseluyko et al., 2022), besides the whole process being time-consuming, especially in the case of large BIM models (Khalili, 2021).

Additionally, despite previous research considering favourable outcomes regarding performance, effectiveness and ease of use of immersive interfaces for the AECO sector (Alhalabi, 2016; Azhar, 2017; Fogarty et al., 2018), these domains are contained within a much broader construct – usability. However, usability testing references to standards or best practices are largely absent from the literature concerning VR research applied to the AECO sector. Therefore, this study includes a comprehensive usability assessment methodology to validate the suitability of the proposed solution to meet the requirements of the AECO maintenance and facility management operational staff is fully described.

The research approach used in this study is based on the design-science paradigm of information systems (Hevner et al., 2004).

According to Simon (1996), design is concerned with how things should be, the development of artefacts and the goals they should attain. Also, according to the author, artefacts are not independent of natural laws, yet they are moulded to human intents (Simon, 1996).

This research also leans on the guidelines discussed by Hevner et al. (2004) and the methodology presented by Peffers et al. (2007).

Hevner et al. (2004) contend that design science strives for utility, whereas behaviour science strives for truth (i.e. justification of theories that describe or predict phenomena), although both are indivisible.

Design science research (DSR) focuses on developing problem-solving artefacts with a clear contribution to the knowledge base (Hevner et al., 2004). Moreover, DSR does not provide optimal solutions; alternatively, this approach generates an acceptable solution to a practical problem framed by an articulated business need (Hevner et al., 2004).

Figure 1 summarises the framework for implementing the research approach, comprising six steps consistent with Peffers et al. (2007). Similar problem-based approaches to AECO-identified challenges can be found in the works of Ding et al. (2017), Pradeep et al. (2021) and Schimanski et al. (2021).

The proposed research framework, namely, the third stage presented in Figure 1, entails the development and validation of a practical solution, which is divided into three stages: file preparation, end-user interactions and update of the original IFC followed by validation according to a user-defined information delivery specification (IDS). The three stages consist of complementary steps for the solution implementation, to be handled by different users, i.e. designers and BIM technicians (users Type 1 – UT1) and buildings’ facility management operational staff (users Type 2 – UT2).

Furthermore, and corresponding to the fifth stage of Figure 1, the proposed solution was subject to a detailed evaluation procedure comprising two assessment phases. During the initial stages, formative evaluations to gather feedback on possible usability problems were conducted, followed by summative evaluations throughout the near-finished development phases.

The following sections thoroughly describe the proposed solution workflow, followed by the usability assessment approach.

The proposed system is centred on the IFC schema for data transfer and interoperability and provides a module for validation of user changes to the IFC file – through inputs made in the VR environment – against custom IDS comprising simple requirements (buildingSMART/IDS, 2022; BuildingSmart, 2022).

The .ids format is based on XML and contains a list of information specifications that BIM objects must comply with buildingSMART/IDS (2022), Tomczak (2021). Current open-source toolkits such as IfcOpenshell are compatible with the IDS standard and allow user-defined specifications (Krijnen, 2021).

The proposed system architecture comprises two fundamental components and one complementary module:

1. a Python module, i.e. a custom widget developed to provide a straightforward openBIM approach to import BIM models into a game engine while maintaining models’ semantics and geometry. This module also updates the original IFC file automatically to match changes made in the immersive VR environment by the end-users. Furthermore, the Python module provides validation of new information added to the model through the use of IDS; and

2. a Unity module comprising a VR immersive interface to access and edit BIM non-geometric information through the use of gesture and voice commands.

The remaining module encompasses a C-Sharp (C#) script using the xBIM toolkit (open-source) (Lockley et al., 2017) so that the final updated IFC file can be converted to COBie spreadsheets.

An overview of the entire workflow is presented in Figure 2 (access to key scripts will be provided on request).

The system is designed for two types of users, designers and BIM technicians (UT1) and buildings’ facility management operational staff (UT2). Therefore, UT1 are responsible for preparing the model to meet the requirements to be imported into the game engine. The preparation entails establishing a JSON file with the necessary information to be accessed by UT2 and converting the IFC file to COLLADA (.dae) so that the geometry and materials of the BIM model can be maintained within a Unity scene. This phase is henceforth identified as the preparation phase. Afterwards, UT2 are tasked with editing the required building elements’ semantic information according to work carried out on-site and to information specifications. This phase is designated as the end-user phase.

The preparation phase is the entry point of the workflow and comprises two semi-automatic steps conducted by UT1:

1. conversion of the BIM model geometry; and

2. setting up a JSON file containing the model semantics (a subset of the original IFC file).

The model geometric information is imported into a Python widget by selecting the “IFC to COLLADA” button (Figure 3 – I), which enables UT1 to choose the corresponding IFC file to be converted to COLLADA.

The conversion process uses IfcConvert, an IfcOpenShell library wrapper, to convert the file to .dae format. Additionally, the process maintains the GlobalId attribute of each building element instead of the element’s name. This feature allows the workflow to be independent of BIM authoring tools’ namings for building elements. Hence, using GlobalIds as an alternative provides a straightforward approach to matching corresponding non-geometric information, ensuring that most geometry and materials of the IFC file are preserved throughout the conversion and import of the .dae file into Unity (Figure 4).

To filter the original IFC into a subset of its entities to be displayed in the virtual environment, UT1 must select the Python widget’s “IFC to JSON” option (Figure 3 – II). In detail, the IFC schema is reduced to a number of elements hierarchically related by type, i.e. a subset of a chosen “IFC type class” entity (IfcOpenShell API Documentation, 2020) (e.g. IfcProduct, IfcBuildingElement), and then converted to the JSON format.

The “User phase” is designed to be performed by operational staff (i.e. UT2) with years of practical experience in on-site maintenance work, although usually lacking the technical knowledge to convey information related to the performed on-site operations directly to a BIM authoring tool. Hence, to avoid excluding UT2 from the BIM information exchange, an approach based on voice commands and gestures was developed to provide alternative means of interaction with building information in an immersive environment (Figure 5). The interface allows access to the building elements data and affords the option to change parameters or create new property sets. Changes are saved as new JSON files, which are sent to UT1 to update the original IFC file through the Python widget and proceed with the validation phase.

The last phase entails the update of the original IFC and the validation according to a user-defined IDS (Figure 6). The suggested validation is carried out by first defining a custom IDS through the Python widget by choosing the option “Customise IDS” (Figure 3 – III). This step requests the specification of applicable entities and requirements to validate the IFC file against, i.e. names and values of building element entities and parameters (including user-defined property sets).

The automatic verification of the IFC file is performed using the IfcOpenShell library, providing output text and BIM collaboration format files. These files enable the verification of building elements’ compliance with the users’ (UT1) specifications.

As asserted by Benyon (2014), usability evaluations may be divided into two approaches: expert-based and participant-based methods.

Expert-based evaluations comprise a set of specialists in human-computer interaction or usability, able to test a development version of an interface. In turn, these should draw on their experience to assess the interface against general design principles, i.e. heuristics (Benyon, 2014). This approach is also known as formative evaluation and is generally carried out during the early phases of development, where significant changes affecting the system may still be pointed out (Benyon, 2014; Nielsen, 1993; Preece et al., 2002).

The proposed interface was initially subject to formative evaluations to gather feedback on possible usability problems. Also, considering that a group of three to five evaluators strikes a balance between a manageable group of experts and the number of problems that may be found during their assessment (Nielsen, 1993), five experts were selected to assess the proposed VR solution (i.e. two usability professionals, two informatics engineers and one BIM researcher). Therefore, most usability problems were identified before the participants-based evaluations (i.e. summative evaluations) – the second evaluation stage.

During the near-finished development stage, summative evaluations were conducted with 31 facility management professionals. The sample size is based on Montgomery et al. (2000), that contend that for practical cases, a sample size with N equal to or higher than 30 will conform with a normal distribution.

The sample comprises 9.7% female and 90.3% male participants. Ages range from 30 to 63 years old. In particular, ten participants were 30 to 40 years old, 13 were between 41 and 50, 5 were between 51 and 60 and the remaining three were aged between 61 and 63. The sample academic backgrounds distribution comprises 41.9% of the participants describing a higher education degree and the remaining participants (58.1%) showing educational backgrounds comparable to primary to high school education, a technical or professional degree.

As contended by Sauro and Kindlund (2005), to perform a summative evaluation of the usability of a product, it is required to define and measure a set of metrics. As such, the assessment of usability attributes (or domains) may comply with international standards and recommendations. This study is consistent with the view of international organization for standardization (ISO) 9241–11 (ISO 9241–11 Ergonomics of human-system interaction — Part 11: Usability: Definitions and concepts, 2018) and ANSI INCITS 354–2001 (ANSI INCITS 354–2001 Common Industry Format for Usability Test Reports, 2001), which define the dimensions of usability as composed of efficiency, effectiveness and satisfaction. Furthermore, this study considers the definition of the referred usability attributes as the following:

• Effectiveness: The extent attained by users fulfilling and completing with precision a prespecified set of tasks and goals [consistent with (ANSI INCITS 354-2001 Common Industry Format for Usability Test Reports, 2001)].

• Efficiency: Resources consumed to attain a specified result leading to a certain level of performance [consistent with (ISO 9241-11 Ergonomics of human-system interaction — Part 11: Usability: Definitions and concepts, 2018; Nielsen, 1993)].

• Satisfaction: The pleasantness of using the system compared to the user’s expectations and needs before the user experience [consistent with (ISO 9241-11 Ergonomics of human-system interaction — Part 11: Usability: Definitions and concepts, 2018; Nielsen, 1993)].

As effectiveness and efficiency are usability domains related to task definition, this study considered effectiveness as the completion rate of tasks (per cent) and efficiency as the time used to complete each task (seconds). Moreover, assuming that each user intends to complete the established tasks, in the event of failure or non-completion of a given “sub-task”, a penalty should be imposed on the completion rate, i.e. a per cent is deduced to the overall task completion rate (Sauro and Kindlund, 2005).

Regarding the satisfaction domain, various questionnaires may be used (e.g. software usability measurement inventory; [Kirakowski, 1995], post-study system usability questionnaire; [Lewis, 2002], system usability scale [SUS] [Brooke, 1996]). One of these examples is the SUS (Brooke, 1996), which is a well-established tool and has the advantage of being relatively short (i.e. ten questions) with proven reliability (Bangor et al., 2009).

This study applied a European Portuguese adaptation of the SUS in agreement with Martins et al. (2015) to ensure that all users could fully understand each question.

Additionally, a qualitative approach was deemed necessary to understand the perceived utility of the developed VR solution. Thus, two semi-structured interviews were conducted to guarantee that the participants’ perceptions and opinions were documented.

Before all the interviews were conducted, participants read and signed an informed consent form to ensure the privacy, confidentiality and anonymisation of the data to be shared (Baptista et al., 2021).

A total of 62 interviews were conducted with 31 AECO professionals, 31 pre-test interviews and 31 post-test interviews between March and May of 2022. It should be noted that both the interviews and the summative evaluation were conducted on the same day, i.e. the first interview was performed, followed by the interface test and, finally, the second interview.

Overall, the summative evaluation comprised an initial interview, followed by a five-minute trial to get acquainted with the hardware in an immersive scenario. Afterwards, each participant was requested to complete a series of four tasks described in Table 1 without the help of the researcher guiding the test. The only information available to the participants was the description of each task and subtasks (read by the researcher on participants’ demand), with no further indication on how to proceed.

These tasks were established by considering earlier feedback from operational staff concerning actual facility management tasks.

Each test finished with a second interview.

After conducting the 62 interviews, they were transcribed, ensuring the anonymisation of the participants’ personal data. All transcribed documents were systematised and analysed using the qualitative data analysis software Nvivo, version 12, based on content analysis as a data-processing technique (Azeem et al., 2012). Also, the same analysis software was used to create a categorical tree with three “mother/main” categories and 17 subcategories prior to the analysis process.

To ensure that most usability problems were recognised before the participant-based tests (i.e. summative evaluations), the formative evaluation consisted in testing the interface by completing a set of three previously established tasks by a group of five experts:

1. go to the living room, select the fireplace and identify the object’s properties;

2. select all slabs using voice controls and create a new property set; and

3. select a glass door using the virtual laser pointer and change the object’s height and length values.

Nielsen’s ten usability heuristics (Nielsen, 1994) were used so that each expert could classify as many usability problems as possible. From the collected feedback, 13 usability issues were highlighted, which were revised later in the system. Most problems concerned “recognition rather than recall”, “help and documentation”, “user control and freedom” and “visibility of system status” based on (Nielsen, 1994).

The data elicited after completing the summative evaluation shows that five participants had previous experience with BIM methodology. In turn, 26 participants reported that they did not know about BIM before taking the test. Another noteworthy aspect is the previous experience with VR equipment: ten participants reported some previous experience, whereas 21 had their first experience with immersive VR equipment during the test session.

The results show that of 31 participants, eight did not complete the test (25.8%), whereas 23 (74.2%) completed the test with 100% effectiveness.

Regarding efficiency, it was necessary to perform a scale transformation since this attribute is measured on a time scale (e.g. the number of seconds spent performing a task), while the others are assessed on a per cent scale between 0 and 100. In this sense, and considering the inexistence of a general rule to establish the duration of tasks (to the authors’ best knowledge), an acceptable target time range, minimum and maximum values were outlined for each of the four tasks. This procedure allows to distinguish the target time for different efficiency ranges and transform each participant’s task completion time into a corresponding efficiency per cent value. Based on a similar approach to Nielsen (1993) and Rideout (1991), two usability goal lines were prepared, as depicted in Figure 7.

The data also indicates an increase in efficiency over the test duration. Figure 8 shows a higher number of participants with 0% efficiency in Task 1 (12 occurrences), following a decreasing number of occurrences in Task 2 (nine occurrences) and a smaller amount still in Tasks 3 and 4 (eight occurrences). This behaviour may be explained by the authors’ intention not to provide prior instructions on how to interact with the system, hence to better assess how participants interacted with the proposed solution without knowing which commands and operations were available to meet the objectives of each task. As such, lower efficiency values were expected during the first task.

Results show that participants were able to interact with BIM information in a natural way using the proposed system, as most participants reached an efficiency level close to that expected for a user familiar with the interface (i.e. high-efficiency values), especially on Tasks 3 and 4.

Concerning the perceived satisfaction of the proposed interface, most participants (64.5%) scored above 68 (i.e. results from answers to the SUS questionnaire), which according to Sauro (2011), is above average. However, the author also suggests converting the SUS score to a percentile rank (Sauro, 2011). Therefore, the results from the participants’ answers to the questionnaire are presented in Table 2.

Another relevant aspect that should be stressed is the consistency among high SUS scores and high-efficiency values. That is, users who graded the system with a greater SUS score also achieved higher performance during the hands-on test. The same is true for users scoring lower in the SUS questionnaire as they achieved lower performance.

From the qualitative assessment, eight emergent subcategories arose throughout the analysis process of the participants’ discourses. According to content analysis, subcategories may emerge from the information shared by the interviewees and can be used to complete the categorical tree previously thought by the researchers (Bardin, 2011; Krippendorff, 1980; Stemler, 2000). Figure 9 presents an overview of the categories and subcategories considered in the final categorical tree.

The categorical planning resulting from Nvivo allowed the organisation and analysis of the information in discourse references attending to the categories and subcategories achieved. In this sense, the presentation of the qualitative analysis of the data is organised into two distinct but complementary stages:

1. the first stage refers to the direct relationship between categories/subcategories and the number of discursive references associated by the interviewees; and

2. the second stage seeks to understand a possible analytical relationship between categories/subcategories: 2. Applicability pre-test and 3. Applicability post-test.

A thorough analysis of each category, as well as comparisons between subcategories, is described in Table 3.

As a final remark, it should be noted that of the only two interviewees who responded negatively to the possible positive impact of this solution in their work context, only one changed his mind entirely after the test. That is, during the second interview, one interviewee answered affirmatively to all Subcategories 3.1.1. – It is possible to implement the proposed VR solution, 3.2.1. – The proposed VR solution is useful and 3.3.3. – The proposed VR solution could be easy to use after some training. The first interviewee who answered negatively about this solution’s possible positive impact in their work context only expressed having changed his perception regarding Subcategory 3.1.1. – It is possible to implement the proposed VR solution, stating that it would be possible to apply this solution at his workplace.

According to the results, the framework and subsequent prototype allow users with no previous knowledge to interact with BIM models, even those lacking a complete understanding of BIM methodology or experience with BIM authoring tools (83.9% of participants). The fact that most participants completed the test without prior instructions on how to interact with the system reveals the ease of use of the proposed solution, thus acting as a more natural approach to accessing construction project information.

The acceptance and suitability of the system, based on the results attained, are considered positive. Indeed, perceived satisfaction demonstrates a general liking and consent for applying the proposed approach to facility management tasks, even among participants with no previous VR or BIM experience.

Another significant aspect that should be stressed is that higher test dropout or inefficiency rates were confirmed among participants with educational backgrounds comparable to primary to high school education or a technical or professional degree. Moreover, among the eight participants who did not complete the test, six were 50 years old or older, while five of these participants had lower academic backgrounds. This occurrence could suggest that more training and dissemination about interaction with immersive VR equipment is required among people with such a profile (i.e. users with lower academic backgrounds and/or older age).

Some highlights of the results concerning a possible analytical relationship between Categories/Subcategories: 2. Applicability pre-test and 3. Applicability post-test – second stage of the qualitative analysis – are worth mentioning. In particular, four of six interviewees who had responded negatively during the first interview to the idea that the proposed solution could not facilitate their tasks changed their opinion during the second interview. Additionally, three of the initial six interviewees identified this solution as useful during the second interview. Also, three of these six interviewees acknowledged the proposed system as easy to use, while two stressed that this solution could be easy to use after some training.

Given the advent of Construction 4.0 and attending to the slow full-fledged acceptance of the BIM methodology as opposed to early expectations (Alreshidi et al., 2017; Walasek and Barszcz, 2017), the authors reflect on the relevance of developing innovative interfaces that are more attuned to the tasks, requirements and working environments of AECO professionals. Thus, this study intends to understand the role of BIM-based NUIs and users’ empirical and tacit knowledge in increasing the level of information (LOI) of BIM models. In particular, semantic enrichment is applied by increasing BIM models’ LOI using gesture and voice interactions within a BIM-based immersive environment. Also, a framework is presented encompassing an openBIM data transfer and storage system, followed by a comprehensive description of the system’s suitability for the AECO sector consistent with international usability standards and previous research recommendations.

From the elicited results, it can be presumed that voice and gesture interactions supported by a BIM-based immersive environment allowed most users to naturally grasp how to interact with building information to complete the required tasks without previous training. Therefore, it can be suggested that the suitability of the proposed system supports the professional’s needs and their tasks within the AECO sector.

In detail, the usability assessment procedure comprised formative and summative evaluations, as well as the feedback gathered from 62 interviews with 31 AECO professionals.

The formative evaluation allowed for major changes to be detected during early development stages. From the feedback of a group of five experts, 13 usability issues were highlighted, which were revised later in the proposed system.

Concerning the summative evaluations and from the elicited results, eight participants did not complete the test (25.8%); however, 23 (74.2%) completed the test without prior instructions on how to interact with the system, most scoring between B and A+ in the SUS questionnaire, which is consistent with their performance. Additionally, participants were able to interact with BIM information in a natural way, as most managed to reach an efficiency level close to that expected of a user already familiar with the proposed interface (i.e. high-efficiency values).

The qualitative assessment provided an in-depth analysis of individual discourses and the comparative relationship between the categories and subcategories related to the pre-test and post-test interviews. Additionally, the relational proximity with the interviewees, guaranteed through a qualitative methodology (Denzin and Lincoln, 1994), made it feasible to reach four key conclusions:

1. Knowledge about VR technology is still commonly associated with recreational and leisure/gaming spaces and moments;

2. Knowledge of VR solutions applied to maintenance is not very common within the sector;

3. The application of the proposed VR solution in the maintenance and facility management field is seen, by most of the interviewees, as having a strong positive impact, being possible to implement in their work contexts, useful and easy to use;

4. There are still challenges to be overcome, among which:

   • A lack of training on the use of this type of solution;

   • The fact that most participants are more familiar with the practical, physical and presential type of work than the use of digital tools; and

   • AECO professionals resist understanding how this solution could be integrated into their daily work without adding extra time to perform their tasks.

Regarding future works, improvements could be made to the presented openBIM framework to streamline the connection towards an immersive digital twin interface. Such an interface would provide accurate information for planning, training or facility management operations.

Furthermore, it would be valuable to use a similar approach to enrich BIM models in case inconsistencies are detected on-site that could compromise the execution of maintenance tasks.

The presented openBIM framework is also prone to be used for semantic enrichment through AR applications. The same modules (i.e. Python and Unity modules – see Figure 2) could convey semantic and geometric information to a game engine handling the interface and interactions designed for AR.

The first author would like to acknowledge the PhD scholarship 2020.07329.BD awarded by Fundação para a Ciência e a Tecnologia (FCT), cofinanced by the European Social Fund (ESF) through the Programa Operacional Regional Norte. This work was financially supported by: Base Funding – UIDB/04708/2020 of the CONSTRUCT – Instituto de I&D em Estruturas e Construções – funded by national funds through the FCT/MCTES (PIDDAC). This work was supported by the strategic programme Centre for Philosophical and Humanistic Studies (CEFH) – UID/FIL/00683/2022.