Safety Competency in an Online Introductory Science Laboratory Course Free

While public health, biosafety, and other industries have developed robust competency domain frameworks that define safety competency through knowledge, skills, and abilities (KSAs), undergraduate laboratory safety does not have such a framework. We propose an outline of safety knowledge (Table 1). Knowledge can be evaluated at multiple levels: factual, conceptual, procedural, and multidimensional (Krathwohl, 2002). Factual knowledge is the basic terminology and details needed to solve a disciplinary problem. Conceptual knowledge is the understanding and interpretation of classifications, principles, generalizations, theories, models, and structures. Procedural knowledge is knowledge of disciplinespecific skills and techniques and knowledge of the criteria for determining when to use specific procedures. Multidimensional knowledge adds a dimension of awareness of one’s own learning and understanding, such as contextual and conditional knowledge and strategic knowledge.

This was an exploratory study to provide insight into the problem of safety competencies with students performing laboratory work outside of the teaching laboratory using mailorder laboratory kits.

Portions of existing instruments (Al-Zyoud et al., 2019; Kocak, 2019; Marin et al., 2019; Reniers et al., 2014) were combined and adapted to measure safety knowledge, attitudes, and practice in an online undergraduate chemistry laboratory setting. Two novel questions were added beyond those modified from existing instruments. The resulting instrument contained 31 items.

Group analysis was performed to collect instrument items into six proposed subsections, with the first subsections covering functional and procedural knowledge aligning with the first three dimensions of the proposed framework for safety competency. The instrument also contained two subsections on attitudes, one addressing attitudes toward safety and the other toward the institution. A final subsection of the instrument collected selfreported safety behaviors. To address left to right bias and positive to negative bias, the rating scales were aligned by placing negative responses on the left in horizontal scales in the online survey platform. The survey questions were closed, with question types including dichotomous questions, semantic differential scale, and multiple choice.

Student responses to specific safety-related questions on prelab quizzes were included in this study. Data was collected in aggregate,with no individual student responses collected. Note that the n will vary for each prelab assessment item as the questions were pulled from pools at random and some questions appeared on more prelab assessments than others due to the broad applicability of the question pool

In this study, survey and prelab questions were posed at factual, conceptual, and procedural knowledge levels aligning with our proposed safety knowledge framework. The levels were evaluated and confirmed by a subject matter expert on the research team.

The survey instrument was administered anonymously via SurveyMonkey. Identifiable information was not collected. Data collection ranged from October 2020 to May 2021. The survey response rate was 18% (n = 12). Participants did not receive any form of compensation for participation. Response rates to educational surveys and safety surveys varies, but this value is within reported ranges for surveys in these areas (Leung, 2021).

The course studied was an introductory general chemistry laboratory course, taught over a 9-week term. The asynchronous hands-on laboratory component was achieved using mailorder laboratory kits from eScience Labs. Students performed experiments in their own workspaces without direct supervision of the instructor.

Students were provided with a digital lab manual to cover basic laboratory skills and techniques, which included safety-related skills and techniques. Student mastery of content knowledge, laboratory skills, and safety was evaluated through a prelab assessment. The use of a prelab assessment has been shown to increase student responsibility and safety communication (Alaimo et al., 2010). Before completing laboratory work, students should know how to integrate useful and factual safety information into a competency, which would allow students to apply safety in new situations.

The students were required to earn a passing grade to turn in the postlab assessment. Ideally, this requirement discouraged students from completing the experiment without having the minimum necessary knowledge, skills, and abilities. Students were given three attempts on the prelab.

Survey responses were exported from the survey platform to an Excel spreadsheet for analysis. Frequencies and descriptive statistics were calculated, including mean, standard deviation, minimum, and maximum values.

The following section discusses the safety knowledge and skills of students performing laboratory experiments using mail-order laboratory kits.

Knowledge was explored in several areas, using both self-reported perspectives and assessments of mastery, and was evaluated at multiple levels. Researchers expected to find evidence of factual, conceptual, and procedural safety knowledge; multidimensional knowledge was not expected to be developed in a 100-level laboratory course.

For the safety awareness dimension of our proposed Safety Knowledge Framework (Table 1), students should possess factual, conceptual, and procedural knowledge regarding chemical and physical hazards. Overall, students self-reported a strong sense of safety awareness through the survey, with only one question showing a lack of factual knowledge related to safety awareness (Figure 1). Approximately 25% of students were either unsure or disagreed with the notion that students frequently hanQRDEe hazardous chemicals in the laboratory. This finding is supported by the assessment results, with 17% of students (n = 6) incorrectly indicating that all chemicals used in this lab course are benign with negligible health effects upon exposure. Evaluating this in the assessments, students provided fully correct responses in identifying physical hazards present in a specific experiment 72% of the time (n = 41), though this was a multipleanswer question and most students identified most of the physical hazards. Providing further evidence of a deficit in conceptual knowledge, when tested on the route of exposure for specific chemical hazards, a concerning number of students could not answer correctly (43%, n = 47).

Despite multiple sources of information on chemical and physical hazards, there are still some students who fail to demonstrate factual knowledge regarding safety awareness. This suggests that the current methods to communicate and promote safety awareness could either be ineffective in relaying the chemical hazards to the students, or a human factors issue with the students themselves disregarding the hazards. Resolving these issues is critical because without factual knowledge, students cannot achieve conceptual or procedural knowledge and thus would not be able to hanQRDEe chemicals according to their hazards. Inability to describe chemical and physical hazards present indicates a possible misinterpretation of the risks involved, which could result in lower PPE compliance (Reniers et al., 2014), incorrect risk interpretation (conceptual knowledge), and an inadequate plan to execute a procedure (procedural knowledge).

For the hazard communication dimension of our proposed Safety Knowledge Framework, students should possess factual, conceptual, and procedural knowledge regarding the means used to convey hazards present, including signage and labeling. To evaluate factual knowledge, the survey and prelab assessments asked students to correctly identify the type of hazard represented by six GHS pictograms.

There is an interesting disparity between the survey and prelab (Table 2). Because the survey was no stakes and the prelabs were low stakes, it is possible that students were more careless in their responses. One could imagine that a student would look up the pictogram to ensure a correct response for the quiz but would not have similar motivation for the survey. This suggests that factual knowledge may be weak as the hazards represented by pictograms are not committed to memory.

To address conceptual knowledge, one prelab question asked students to identify the appropriate pictogram when working with a specific chemical. Only 77% of respondents (n = 70) correctly identified the appropriate pictogram, indicating inadequate conceptual knowledge of hazard communication.

The prelab assessments provide further evidence of factual knowledge predominating over conceptual and procedural knowledge related to hazard communication. Most students could correctly identify the two GHS signal words (93%, n = 14), a measure of factual knowledge. When queried on conceptual knowledge, correct answers to reporting signal words for specific reagents used in an experiment, correct answers were provided 67% of the time (n = 155). Similarly, when asked to identify the reagent with the fire hazard based on chemical labeling, correct answers were provided 29% of the time (n = 10). A weak level of procedural knowledge was also identified regarding hazard communication, where 58% (n = 61) did not correctly report the labeling for a secondary container for a specific chemical in an experiment. These results together show a need for bolstering conceptual and procedural knowledge related to hazard communication.

The “chemical use and emergency response” dimension of safety knowledge has several key elements to explore. Regarding accident and incident response, students should possess factual, conceptual, and procedural knowledge regarding emergency response equipment and procedures for responding to accidents and incidents. A gap between confidence in knowledge and demonstrated knowledge is important to understand related to safety as a false sense of confidence can lead to accidents. To evaluate factual knowledge, the survey asked students about their knowledge of the location of emergency safety equipment, including sinks, showers, fire extinguisher, and first aid kit, to which students unanimously reported confidence that they knew the location of all listed emergency equipment items. However, when this knowledge was tested on the prelab, less than perfect conceptual knowledge was demonstrated. For example, when asked to correctly identify the appropriate fire extinguisher given a chemical to be used in an experiment from the lab kit, 88% (n = 84) answered correctly.

Similarly, students unanimously reported confidence in responding to incidents (chemical spill to a work surface or floor) and accidents (chemical spill with skin exposure and chemical spill with eye exposure). Regarding incidents, their confidence was matched with a 100% response rate correctly indicating that clean-up procedures would follow the chemical’s SDS and thus demonstrating factual knowledge. Similarly, students correctly indicated they would follow the SDS measures for responding to an accident where a chemical exposure to the eye occurred. However, there was a lapse in factual knowledge related to skin exposure, with 17% of respondents incorrectly stating they would place the exposed skin area under running water. Overall, the survey responses indicate a strong foundation for responding to a chemical spill. However, when given a specific spill scenario using a chemical from their laboratory kit to evaluate conceptual knowledge, only 76% (n = 77) of students correctly identified the appropriate spill clean-up measures. This suggests that students broaQRDEy have factual knowledge of accident and incident response, but there is room to improve conceptual knowledge. While it is still possible for students to possess procedural knowledge for emergency response, the factual and conceptual knowledge are critical to mounting an appropriate response. These knowledge deficiencies suggest that there is a need for more effective teaching of accident and incident response procedures.

Hazard control is an element within the “chemical use and emergency response” dimension of safety knowledge, divided into three areas: personal protective equipment (PPE), engineering controls, and administrative controls. Our results provided the most information about PPE. Regarding PPE, students should be able to describe PPE and risk reduction as well as use PPE to mitigate hazards. The survey asked students to honestly report their actual safety behaviors (Figure 2). Reassuringly, all students reported using proper PPE when working with chemicals. This is important as they are operating from their own spaces, outside of direct supervision of lab personnel. However, this reported behavior should be supported with evidence of factual, conceptual, and procedural knowledge.

In the prelab assessments, students unanimously demonstrated strong factual knowledge by connecting route of exposure to appropriate PPE. Recall, students struggled with identifying specific routes of exposure for specific hazards (safety awareness dimension). In an evaluation of conceptual knowledge, students were asked to identify appropriate eyewear for hanQRDEing specific chemicals. A concerning 22% (n = 20) could not correctly identify the appropriate PPE. This is a clear example of a disconnect between factual and conceptual knowledge.

Interestingly, there are also examples where procedural knowledge was stronger than conceptual knowledge. When asked about the permeability of disposable nitrile gloves on the survey, 15% of students (n = 5) incorrectly stated that the gloves were impermeable. Despite this, 100% of students correctly reported that the disposable nitrile gloves were not reusable once removed. Students also unanimously (n = 24) understood that their PPE should not be worn when leaving the work area (e.g., during restroom breaks). This survey data on factual knowledge is supported with prelab data, where 100% (n = 38) correctly identified appropriate footwear for laboratory work, though the factual knowledge related to clothing used when working with open flames was not as strong (17% incorrect, n = 6).

Based on the survey response, it may be worth addressing proper clothing with students for each round of experiments to improve execution of that safety behavior. In a traditional laboratory setting, a lab supervisor can identify students wearing inappropriate clothing and prevent them from engaging in chemical work until their clothing is appropriate for the hazards present. This level of supervision is not possible with mail-order laboratory kits, especially when the course is asynchronous online with globally distributed students in multiple time zones.

The regulatory compliance element of safe chemical use and emergency response in the safety knowledge framework establishes an expectation to cite relevant regulations, explain how compliance is achieved, and operate in a way to achieve regulatory compliance (factual, conceptual, and procedural knowledge, respectively). While regulatory compliance has many aspects, this study focused simply on hazardous waste disposal. The survey directly asked students about disposal of chemical waste. Unanimously, survey respondents correctly indicated that the disposal method depends upon the chemical identity, suggesting conceptual knowledge. However, the prelab assessment results did not support strong conceptual knowledge. When asked “Which reagents from this week’s lab should not be disposed of in the trash/drain”, 42% of students (n = 22) got this question incorrect.

Previous research of in-person undergraduate chemistry laboratory students shows some weakness in multiple dimensions of safety knowledge (Al-Zyoud et al., 2019; Walters et al., 2017; K. Wu et al., 2021). Based on the results of the study reported here, it appears that similar areas of concern exist with undergraduate students completing laboratory courses through mail-order kits. Because there is potential correlation between safety awareness and safe practices (Abdullah & Abd Aziz, 2020; Walters et al., 2017), interventions should be aimed at addressing gaps in factual, conceptual, and procedural knowledge. Further education and training should be incorporated into the laboratory course curricula, whether in person or online.

This study provided insight into safety competency of students completing introductory undergraduate chemistry through distance education, using mail-order laboratory kits. While this study was intended to be exploratory, the small population size coupled with the low response rate introduces nonresponse error. Given the population size, actual response rate, and an 85% confidence level, the margin of error for the survey data was 19%. These results are not generalizable to other populations. Results of safety culture research are largely not viewed as generalizable, though (Biggs et al., 2010). However, the results of this study are still important in understanding the actionable safety knowledge of online learners.

This survey instrument has face validity established through its review by a safety expert prior to implementation. Because this instrument was developed through minor modifications to portions of existing instruments, a review by a psychometrician for issues like leading questions was not deemed necessary by the researchers. The survey was pilot tested with 10 participants; no questions were flagged for investigation based on the pilot testing. Future work should use a larger sample size to perform confirmatory factor analysis to further validate this instrument developed through modification and adaptation of portions of multiple existing validated instruments.

Response bias is another potential limitation of this study. The survey asks students to self-report safety behavior. There may be a psychological tendency to avoid admitting unsafe behaviors (social desirability bias), even in an anonymous setting without potential for consequences.

While the online chemistry laboratory course used in this study was designed by a safety expert and safety was clearly embedded into the curriculum, the results of the detailed item analysis and direct inquiry of students show that there is room to improve safety training. While safety attitudes are strong, there are some gaps in safety awareness, hazard communication, and safe use of chemicals and emergency response. Specifically, there are some noted gaps between factual, conceptual, and procedural knowledge in these domains. It is important to reach students early so that they can build their safety knowledge over time, with the goal of reaching multidimensional safety knowledge. As with in-person chemistry education, a strong safety culture includes deeply embedding chemical safety education into the curriculum.

Future work should explore correlations between safety knowledge and attitudes for students using mail-order laboratory kits. Furthermore, future work should explore how well awareness, knowledge, and attitudes explain self-reported safety behavior. Because of the reluctance to move laboratory courses to online modalities for fear of impacting skill development, this type of research is necessary to provide evidence of efficacy not just in teaching core course concepts, but also in developing critical safety knowledge, skills, and abilities.

Acknowledgments: We would like to acknowledge the contributions of Dr. Darryl Chamberlain to the organization and editing of this manuscript.