Occupant-Building Interaction (OBI) Model for University Buildings

Building environments are complex and dynamic constructs. With advancement in building automation, these environments are transforming into complex computer systems. Acknowledging this transformation, a new research topic called Human-Building Interaction (HBI) has emerged in the area of Human-Computer Interaction (HCI).

HBI is a very wide research domain and it is difficult to define HBI in one sentence. Nembrini and Lalanne (2017) proposed a detailed definition which explains HBI by comparing it with Ambient Intelligence (AmI), mapping HBI-relevant research fields, and listing various dimensions and modalities in users’ interaction with a building. Following this definition, HBI can also be defined as the uni-directional or bi-directional flow of raw data and/or processed data in real-time or deferred-time between humans (occupants, designers, facility managers, etc.) and buildings (real or virtual) for the benefit of either or both. Existing research in this domain focuses on making use of this interaction to efficiently manage the building facilities. Currently, data is collected from the building and its occupants, analysed and sent back to the building mainly to manage occupant comfort (Li et al., 2017), maintain HVAC control strategies (Abraham et al., 2017) and improve occupant safety (Chen et al., 2018). This paper reflects upon the term Occupant-Building Interaction (OBI) and defines it as bi-directional flow of raw data and processed data in real-time between the building and its occupants; facilitating efficient bi-directional communication. The essence of OBI is to raise bi-directional awareness among the elements of the building and its occupants, so the building is “aware” of its occupants, e.g. in terms of location, and vice versa. This can be achieved through analysing the data collected from the building and its occupants in real time and visualizing it in a digital model.

To this end, first, we simplify the intricacies of OBI and explain the important data types needed for efficient two-way communication between the buildings and their occupants. This includes data related to occupant safety, building consumption, occupant comfort and occupant location. A conceptual hybrid framework is also presented structuring the collection and analysis of the data related to occupant location, i.e. occupancy-data, for real-time location sensing (RTLS) inside the building. Past research on indoor RTLS details the use of various sensors to estimate indoor location, but no or little research has been done which uses a combination of coordinate-data and contextual-data to estimate the location of an occupant inside the building. The proposed framework will collect both coordinate-data and contextual-data of occupants of university buildings and visualizes it in real-time on a digital model. The building management system (BMS) will make use of a computer application (COM) and the occupants will be provided with a dedicated smartphone application (APP) to access the results of this data. It will help the BMS to estimate the number of occupants, track their movement and will help the occupants in indoor navigation.

To improve occupants’ thermal comfort, Li et al. (2017) uses building-data obtained from sensors with occupant-data obtained from wearable devices, to train personalized comfort prediction models. These models can be used to determine optimum HVAC control strategies for building environments. Abraham et al. (2017) used machine learning methods to train the occupant behaviour-related energy data and predict the energy consumption pattern of in a building. The data was measured by using sensors, metering devices, and surveys. A fuzzy rule-based descriptive and predictive model was given by Jazizadeh et al. (2014) to compute the comfort profiles of the building occupants. The research used comfort data from occupants through participatory surveys and temperature data through a sensor network to predict the comfort profiles of the occupants. Agent-based modelling (ABM) is also used for developing an occupant behaviour model from the data collected using various sensor nodes and survey sheets to improve accuracy of energy estimation (Jia et al., 2018). Although these studies do not always share the results of the collected data with the occupants, it is still used for the benefit of the occupants. Similarly, Chen et al. (2018), Chenga et al. (2016), and Cheung et al. (2018) make use of the data to aid the occupants in case of an emergency inside the building. Chen et al. (2018) makes use of simulated raw data from fire dynamics simulators (FDS) to warn the occupants about existing fire inside the building. Accordingly, it provides guidance for the occupants to escape and the firefighters to quickly discover the fire using LED light indicators placed at various locations in the building. Chenga et al. (2016) uses the data from sensors to detect fire and guide the occupants with an escape route using a smartphone application. Cheung et al. (2018) developed a system which alerts the safety managers of the presence of any harmful gases which is detected by a group of wireless sensor nodes placed at various locations in an underground construction site. However, research which encompasses bi-directional and real-time data flow between the building and its occupants for the benefit of both is still missing in the literature.

Indoor RTLS has many advantages and is an important component in establishing OBI. Rüppel et al. (2010) made an integrated navigation system for firefighting purposes in an airport based on Wireless LAN (WLAN), Ultra-WideBand (UWB) and Radio Frequency Identification (RFID). They recommend the use of different positioning technologies depending upon the adequacy of the existing environments in a complex building. A comparison of indoor location sensing (ILS) technologies like Global Positioning System (GPS), Inertial Navigation System (INS), Infrared, UWB, WLAN, and RFID was done by Li and Gerber (2011). According to them RFID, even though with its limitation is the most suitable for ILS when evaluated based on accuracy, affordability and usage. Li and Gerber (2011) also compared three ILS methods Triangulation, Proximity (LANDMARC) and Scene Analysis and concluded that LANDMARC serves the best for building environments. In a more recent research, ILS technologies like WiFi, Bluetooth (BLE), Zigbee, RFID, UWB, visible light, acoustic signal, and Ultrasound were compared by Zafari et al. (2017) and RFID when compared with other ILS technologies was found to have less cost, more energy efficiency, high accuracy, less latency and more immunity to interferences. An indoor navigation system for visually impaired people that senses acoustic signals and voice commands played through headphones is given by Martinez-Sala et al. (2015). It uses UWB for positioning because of its level of accuracy and low installation complexity. All the ILS systems mentioned above estimates the coordinates of the occupants from the sensors. Systems also focus on utilizing contextual data for the purpose of conveying information between buildings and their occupants. Galván-Tejada et al. (2018) makes use of the data captured from sound sensors to recognize human activities and infer the location of occupants from the contextual information that the activity represents. The concept is to use human activities as the context to describe location of occupants inside a building. Other used contextual data received from light sensors (Randall et al. 2007) and geomagnetic signals using the magnetometer sensor in a smartphone (Kim et al. 2012; Han et al. 2012). Zhu and Sheng (2011) developed a hybrid approach using coordinate and contextual data which combines motion data and location information for indoor human daily activity recognition. However, there still exists a need for such a hybrid system to estimate indoor location of building occupants for different purposes. Therefore, we propose a hybrid ILS system which will capture coordinate-data from RFID-based system and contextual-data from sound sensor to accurately capture the position of an occupant inside a building.

As mentioned earlier, the purpose of OBI is to generate awareness in both the building and its occupants, by making them interact or communicate with each other. Figure 1 illustrates the OBI framework with some data types which can help in creating this awareness. The extent and variety of the data that can be exchanged between the two entities to make them interact is increasing. In this paper, we identify the important data-types, which are essential for OBI. To this end, we follow a top-down approach and answer the following two questions: (1) what important information or data the occupant needs to know about the building; and (2) what important information or data the building needs to know about the occupant. To clearly understand these data-types, we introduce the terms processed building data (PBD) for the former and processed occupant data (POD) for the latter. The raw data required to get the processed data (PBD and POD) is captured from the building and occupants through various sensors attached to both. The occupants also contribute to the raw data by taking participatory surveys on their smartphone. The authors also reflect upon the various kind of sensors which can be used to measure these data-types in this research.

There can be various PBD types that can be communicated to the occupants for their benefit. For this research, the authors classify PBD into two important categories, namely processed consumption-data (PCD) and processed safety-data (PSD). PCD represents the overall resource consumption in a building and will inform the occupants if they are making efficient utilization of the resources available. It will include the amount of energy (electricity, gas) and water consumed by the building over a period. The raw data that gives PCD can be captured using electricity meters and gas meters installed in the building. The PCD can be communicated to the occupants through the APP. PCD can help in raising occupants’ awareness about the resource wastage and suggest necessary conservation strategies.

The PSD, on the other hand, is required to raise occupants’ awareness about their safety while inside the building. As such, it can convey information about safety issues inside the building that can include a potential fire, contamination in air or water, structural instability, and security hazards; e.g. in case of fire the collected PSD can be used to alert the occupants about where the fire has been detected inside the building and accordingly guide them to a safe exit route. PSD can also help the occupants in locating and using the firefighting equipment that are at their disposal inside the building. Raw data about fire can be captured using fire sensors and escape route can be found by using a combination of fire sensors and ILS system. Fire alerts, escape route navigation and location firefighting equipment can be communicated to the occupants through the APP. Air and water quality monitoring sensors can provide raw data and the related PSD can be transferred to the occupants through the APP. Any structural damage sustained by the building due to natural or man-made actions can be measured by specialized structural stability sensors and the raw data from these sensors can be analysed and sent to occupants in the APP. The PSD measured using ILS system and sound sensors can also inform the occupants about intruder presence and if there are gunshots fired inside the building premise respectively.

As for PBD, there can be a variety of POD types that can be measured from the occupants and be used for the benefit of the building. In this research, POD is used to help understand the occupants’ comfort as well as buildings’ occupancy details. POD is classified here as processed comfort data (PCoD) and processed occupancy data (POcD) respectively. PCoD includes information about the respiratory, thermal, visual and acoustic comfort levels of the building occupants. The air quality sensors can serve the purpose of acquiring the raw data for PCoD. Raw data from temperature sensors, relative humidity sensors and weather stations serve the best in assessing the thermal comfort of the building occupants and contribute to PCoD. Light intensity sensors and sound sensors can provide the raw data about the light and sound comfort levels respectively. Sensors such as a fitness band can be used to collect physiological and behavioural data from the occupants to determine their comfort levels. POcD can be used to inform the building about the occupant number and occupant movement. Raw data about occupants’ count and their movement can be captured using various ILS systems discussed previously. In addition to the raw data measurement methods discussed above, which make use of sensors installed in the building and/or on occupants, occupants’ response to participatory surveys can also be used as a major contributor to the raw data required for POD.

Following the described principle of OBI, we propose a hybrid RTLS system, which makes use of coordinate data generated from the RFID-based ILS system and contextual data measured from the sound sensors to collect the raw occupancy data in a university building. Figure 2 represents OBI framework for measuring and analyzing occupancy data. RFID was chosen over other ILS systems like WiFi, BLE, UWD, etc. to measure the coordinate data because of its advantages highlighted by Li and Gerber (2011) and Zafari et al. (2017). RFID-based ILS system typically consists of a tag, a reader and a data processing subsystem Li et al. (2011) and it operates at a certain frequency. The tags can be classified as active and passive based on their power source. The passive tags are activated by electromagnetic energy emitted by the reader whereas the active tags require an embedded source of energy to operate. The contextual data in this research refers to the human activities that can describe the location of an occupant inside the building. The sound data originated from different rooms inside a building is used in the proposed system to predict the presence of an occupant there. The dedicated APP on occupants’ smartphone will make use of the smartphone sound recorder to measure the sound data and subsequently predict an occupants’ location inside the building.

University buildings usually are occupied by fixed objects such as furniture as well as moving occupants including students, faculty, staff, visitors, etc. RFID tags will be installed on fixed objects with known coordinate location (reference tags) and will also be given to the moving occupants of the university building (tracking tags). Readers will be installed at the corners of the rooms. The frequency at which the RFID system will operate is Ultra High Frequency (UHF). The number of RFID readers will depend on the room size and the number of tags (reference & tracking) will depend on the number of occupants.

A description of human activity sound data is prepared for university buildings. The data set consists of 17 commonly performed university building activities. The professor and students talk during the classes and there is an abrupt movement of classroom furniture when the class ends as students get up to leave the classroom. Reading room/Library/Offices are mostly quiet and in such an environment, the sound coming from typing on keyboard, using the mouse, turning a book’s page and from printers can be used for activity recognition. The common areas have student discussing and eating. In addition, several doors opening and closing into common areas, as a result, the sound coming from the doors can be used for recognizing the door opening and closing activity. The pantry has sound from the tap water, microwave and coffee machine. The washroom has the sound from tap water, water flush, towel dispenser and hand dryer. Table 1 describes all the activities and the rooms where these activities are mostly performed. Using sound as context for activity recognition is preferred over other mediums because the APP can have constant access to the smartphones’ sound recorder once it gets the permission. Using other smartphone sensors like the accelerometer, gyroscope, etc will have a limitation in recognizing the context of the location occupants are at. Visual data from the smartphone camera can help in identifying the location, but occupants would then have to keep their smartphone camera exposed and running all the time. Use of sound recorder is more accurate, convenient, and energy efficient for ILS when compared with other smartphone sensors.

The designated APP will make use of the smartphone sound recorder for recording the sound data. Sound will be recorded at a standard sampling frequency rate of 44.1 KHz Cho et al. (2017). The duration for which the sound data will be recorded and filtered for further analysis will depend on the activity that takes the least duration to complete one cycle.

The location of tracking tags is estimated by comparing the received signal strength indication (RSSI) of the tracking tags with that of the reference tags which are fixed at known coordinate locations. When the occupants equipped with tracking tags move inside the building, the reader communicates with them and records their RSSI values. The reader does the same for the reference tags. All these values are sent to the data processing subsystem where the RSSI values of the reference tags and tracking tags are compared and the location of tracking tag is predicted. The k nearest neighbour (KNN) algorithm is used for predicting location of reference.

Analysis of sound data will be done by converting it into an integer array, where each integer will represent the magnitude of the sound at a given time (Galván-Tejada et al. 2018). The sound data will then be structured based on the activity that it represents and the cycle time of that activity. The structured data will be imported into MATLAB and to extract the required set of features from the collected data. The statistical features that will be used for identifying the human activities from the structured sound data are Kurtosis of the probability distribution of the integer array, Skewness of the probability distribution of the integer array, Mean of the integer array, Median of the integer array, Standard deviation of the integer array, Variance of the integer array, Coefficient of variation (CV) of the probability distribution of the integer array, Inverse CV, 1st, 5th, 25th, 50th, 75th, 95th, and 99th percentile of the probability distribution of the integer array, and Mean of the integer array after trimming the bottom and top 5% elements Galván-Tejada et al. (2018). A probability curve will be plotted to select the top features for activity recognition of each room in the university building. These features will then be used to train the ML algorithm for activity recognition and subsequent recognition of the rooms where the occupants are located.

The POcD will be represented on a digital model of the university building. The model can be accessed by the occupants and the security personnel, through the APP or the COM. A BIM-based 3D model will offer better visualization than 2D drawings to the occupants and security personnel to navigate and to monitor respectively. In case if a BIM-based model for the building is not available then laser scanners will be used to capture point clouds and a model will be made by making use of these point clouds.

The OBI framework introduced in this research provides required directions for establishing effective interaction between buildings and their occupants. The purpose of OBI is to raise awareness between buildings and their occupants about one another which can result in benefits for both the entities. To this end, the various data types that should be exchanged between the building and its occupant are identified in this research. These data types are categorized as processed building data (PBD) and processed occupant data (POD). PBD and POD are further subcategorized as processed consumption-data (PCD), processed safety-data (PSD) and processed comfort data (PCoD), processed occupancy data (POcD) respectively. The raw data required for analysis and extraction of processed data is collected from both the building and its occupants by using various sensors installed in buildings and given to occupants. The processed data is communicated back to building and occupants through building information modelling (BIM)-based model which can be viewed in a computer system (COM) or in a dedicated smartphone application (APP). A novel conceptual framework is then presented to measure and analyse the occupancy-data for the specific case of university buildings. A hybrid indoor real time location system (RTLS) that makes use of coordinate data and contextual data to determine the location of an occupant inside the building is proposed. The hybrid system will use RFID tags and readers to measure the coordinate data and sound recorder enabled from the APP. Though previous research on indoor RTLS have claimed RFID-based indoor location system (ILS) to be the most suitable among others, it still has its limitations in terms of accuracy. When coupled with sound sensors in a hybrid system the accuracy of assessing the indoor location of an occupant can be improved. The hybrid RTLS system is in its development phase and currently there is not enough data to give results. The use of the APP is subjective to occupants and poses a challenge in the implementation of the hybrid RTLS system.