Participation and satisfaction with thee-learning course “Mental health promotion in school health care”: a mixed-method study Open Access

Globally, the mental health status of school-age children and adolescents has reached alarming levels. The number of people seeking help for their mental health problems is increasing annually (Gyllenberg et al., 2018). Moreover, current events in the world, like COVID-19 and the social isolation it caused, may have further challenged the mental health of children and adolescents (Loades et al., 2020). This escalation can also be seen in schools, where school nurses spend a large part of their working time dealing with pupils’ mental health issues. For example, in Finland, care of mental health-related issues consumed an average of 25% of school nurses’ face-to-face working time (Hietanen-Peltola et al., 2022).

School nurses’ competence in mental health issues has proved to be insufficient (Markkanen et al., 2021) and they need evidence-based education on managing mental health issues (Putkuri et al., 2021; Kaskoun and McCabe, 2022). In our earlier study, we identified those topics needed in continuing education, such as assessment tools, etiology and symptoms of mental disorders, the chain of care paths, preventive interventions, methods of psychosocial treatment and medical treatment (Putkuri et al., 2021.) Similar training needs have also been identified previously for school nurses in the USA (Bohnenkamp et al., 2019). Based on these identified competence needs, an e-learning course with three modules was developed to improve the competence of school nurses in mental health issues.

Research on mental health training for nonpsychiatric-specialist health workers is scarce; a systematic review found only 29 studies with relevant training courses (Caulfield et al., 2019). Research on mental health training for school nurses is even more limited; we found only three training interventions with research reports. These were the Mental Health Training Intervention for Health Providers in Schools “MH-TIPS” (Bohnenkamp et al., 2019; Bohnenkamp et al., 2022), a Quality Improvement Evaluation for School Nurses and Teachers “QUEST” (Haddad et al., 2018) and The Child Anxiety Learning Modules “CALM” (Muggeo et al., 2017). The first, MH-TIPS, is targeted at the competence of school nurses to address student mental health concerns, the second, QUEST, at school nurses’ attitudes toward and their knowledge of and recognition skills as regards depression. The third, CALM, improves school nurses’ ability to administer interventions that will reduce the anxiety symptoms of school children. The lack of studies investigating mental health training for school nurses indicates the need for more studies in this area.

For this study, there was also a need to develop a new, fully online, short training course for school nurses. Of the training courses mentioned above, two were blended learning: online learning was combined with time-bound in-person training and consultation (Bohnenkamp et al., 2019) plus extra printed and audio materials (Haddad et al., 2018). One training course (Muggeo et al., 2017) was completely in-person. This is somewhat surprising considering the popularity and advantages of fully online education (i.e. e-learning). One of these advantages is flexibility, due to the learning not being limited by space- or time (Regmi and Jones, 2020). Moreover, the trainings were quite extensive and took several days (Bohnenkamp et al., 2019; Haddad et al., 2018), even though time-consuming activities are one of the factors identified as increasing the dropout rate (Longhini et al., 2021). However, the satisfaction of learners with these previous training courses has been high (Muggeo et al., 2017; Bohnenkamp et al., 2022), which encouraged further development of the course examined in this study.

There is a need for studies concerning mental health training for school nurses, especially related to brief e-learning courses. To fill this gap in the research, we conducted a mixed-method study to evaluate the participation and satisfaction of learners in the developed e-learning course and to describe any related factors.

The Mental Health Promotion in School Health Care – e-learning course was developed to improve school nurses’ competence in mental health issues. Merrill’s first principles of Instruction guided the development. According to these principles, training should:

• be based on real-world problems;

• activate the learner’s prior experiences;

• provide demonstrations of the subject matter in addition to mere information;

• support the application of learned; and

• help integrate the learned skills into practice (Merrill, 2002).

Adhering to these principles improves satisfaction and learning outcomes of learners compared to conventional methods (Badali et al., 2022). The quality indicators of postgraduate medical e-learning (de Leeuw et al., 2018) were taken into account when developing the course, and most of the indicators were attained. In Supplementary file 1, a detailed description of the development phase is provided and reported according to the TIDieR checklist (Hoffmann et al., 2014). The developed course consists of three modules:

1. “Prevention of mental health problems in school health care.”

2. “Mental health problems in school health care.”

3. “Depression, anxiety, and suicidality in school health care.”

Each module has a duration of approximately 45 min. The learner could complete one or more modules, in any order. The course was published in February 2021 in Duodecim Oppiportti e-learning portal which is a Finnish portal of Continuous education for health-care professionals published by Duodecim Publishing Company Ltd.

We conducted a convergent mixed-method study with a descriptive, posttest-only design. The information obtained from the quantitative and qualitative data was combined after analysis, with the aim of gaining a more complete understanding (Creswell and Plano Clark, 2018). The framework of Moore et al. (2009) guided the formulation of the research questions. According to this framework, originally developed for continuing medical education, the learning outcomes can be divided into seven levels:

1. participation;

2. satisfaction;

3. knowledge;

4. competence;

5. performance;

6. patient health outcomes; and

7. health status of the community (Moore et al., 2009; Zaghab et al., 2015).

In this study, we focused on the first two levels, which are the fundamental basis for the following ones. (Table 1)

There were two different samples in the study: Sample 1 for quantitative and Sample 2 for qualitative data. The participants for the first sample were selected with a nonprobabilistic sampling from the learning portal: every learner answering the feedback questionnaire was included (Creswell and Plano Clark, 2018). The participants for the second sample were selected with purposeful sampling. According to the study by Hennink et al. (2019), over 90% of codes can be identified and code saturation is reached in four focus groups. Thus, our goal was at least four groups.

The study was conducted in Finland, where the majority of school nurses have completed the four-year education program for public health nurses at a University of Applied Sciences (UASs) and have been awarded a Bachelor of Health Care in Public Health Nursing (Government Decree 1129/2014). Finnish school nurses provide high-quality prevention and health promotion services including periodic individual health check-ups and parental support (Grym and Borgermans, 2018). One school nurse is responsible for an average of 532 pupils (Hietanen-Peltola et al., 2022).

In the quantitative section, data was collected from the learning portal between February 2021 and November 2022. Part of the data was collected directly from learning portal and part via a feedback questionnaire (Table 1). The learner was asked to give their profession when registering as an Oppiportti user and nonregistered learners were asked when giving course feedback. Feedback was requested at the end of each module, as a part of the normal usage of the learning portal. The feedback questionnaire (Supplementary file 2) included 10 questions; in this study, we used quantitative replies that answered our research questions (Table 1).

In the qualitative section, data was collected in four focus groups in May 2021 and February 2022. The aim was to deepen the understanding already achieved through the quantitative data (Table 1). The invitation to participate was published in March 2021 on the Facebook page of the Public Health Nurses’ Association, in the Association’s newsletter and in the Facebook group. Moreover, the researcher (T.P.) invited public health nurse students from two different UASs to participate by briefly visiting their lessons and posting an invitation. Interested people were directed to complete the electronic background questionnaire, provide written informed consent and to leave their email addresses. They were free to do modules at any time suitable for them. The first author (T.P.) contacted them by e-mail and offered time for an interview by means of remote access. T.P. conducted 35–60 min focus groups with two to six participants. The discussions were audio-recorded and transcribed verbatim.

Quantitative and qualitative data were analyzed separately (Creswell and Plano Clark, 2018). Quantitative data were analyzed using SPSS 28. We used descriptive statistics such as frequency, percentages, mean and standard deviation. The Mann–Whitney U test was used to assess the differences between public health nurses and other learners.

Qualitative data were analyzed using nVivo 12. Data from four focus groups were combined and analyzed with inductive content analysis. Initially, all phrases reflecting participants’ perceptions of the course were selected and then these meaning units were coded, grouped and categorized. Subcategories and categories were named by describing their content (Elo and Kyngäs, 2008). The process was conducted by the first author, and the other authors participated in discussions concerning the process and results to achieve consensus and to ensure confirmability.

The study was conducted following the WMA Declaration of Helsinki (WMA 2022) and the national guidelines of The Finnish Advisory Board on Research Integrity (TENK 2019). The COREQ checklist (Tong et al., 2007) guided the reporting of the study (Supplementary file 3). A statement from the ethics committee was not necessary in the Finnish context, because participants were professionals or students, and the topic was not sensitive. In the qualitative section, participants gave their written informed consent to participate in the study. Research data management was fulfilled according to the guidelines of the Finnish Social Science Data Archive (FSD) (2022).

In the quantitative part, there were 690–954 participants per module. The number of responses varied slightly according to the module and the question (Tables 2 and 3). The best response rates were in the first module “Prevention of mental health problems in school health care” and in the 3rd module “Depression, anxiety, and suicidality in school health care”. The second module “Mental health problems in school health care” also had a good number of responses, although there were also many missing answers.

In the qualitative part, the participants were comprising of two school nurses (actual target group) and 12 public health nurse students (potential members of the target group in the future without current working experience as school nurses). The demographic information on four participants is missing. Ten of the public health nurse students had a degree qualifying them as a registered nurse. The working experience of these 10 public health nurse students as health care professionals ranged between 4 months to 12.5 years (mean 6 years). Their age range was between 22 and 45 years old (mean 33 years), and all had performed courses previously in the Oppiportti learning portal. Three participants had also completed some continuous learning concerning mental health.

The three modules have opened a total of 12 922 times during the one year and nine months period (12.2.2021–17.11.2022). The module most opened was the third module with 6,517 openings. In all three modules, a fairly large proportion (23.5%–35.9%) of the logged-in registered learners or learners answering the questionnaire were public health nurses (Table 2).

Qualitative results from the focus groups supplemented the quantitative results and provided information about factors influencing participation within the course. The identified factors were favorite methods attracting attention, the relevance of additional information and postprocessing boosting diligence (Figure 1).

Based on the interviews, some of the participants favored certain learning methods and focused mainly on these methods when completing the course. Some liked to read texts, while others thought that they would learn better using the infographics. Some favored quizzes, while others felt that different drag-and-drop-tasks were satisfying. More audio content was requested, for example, recorded microlessons. The videos produced mixed opinions: some had completely skipped them, while others thought they were the best content in the course:

The modules included hyperlinks to additional information available either by clicking on additional information windows or as links leading to material outside the course. Learner’s experience of the relevance of additional information determined how comprehensively it was used. Some of the interviewees liked the opportunity to read more and to find relevant Web pages and publications about interesting topics. Others had not opened the external links at all and preferred all the necessary information to be included in the course:

Learners’ awareness of postprocessing seemed to boost diligence. The participants knew that after the course they would have an interview about its content as a part of the research project. Generally, the content of the course was comprehensively reviewed. However, some had skipped pages and some said they had pressed every click to obtain the completion credit but had not read this additional information. Some reported that they had completed the course particularly carefully because they knew they would be attending the focus group afterward:

The overall satisfaction of the public health nurse learners with the e-learning course was high; in the first module, the mean score (scale 1–5) was 4.12 (SD 0.74), in the second the module 4.30 (SD 0.79) and in the third module 4.00 (SD 0.87). There were no significant differences between the public health nurses and other participants (Table 3).

The public health nurse learners scored high in all three satisfaction areas:

1. development of competence (mean 2.4–2.6, scale 1–3);

2. motivation for learning (3.67–3.76, scale 1–4); and

3. application of learned in practice (3.11–3.26, scale 1–4).

The public health nurses scored the last two of these areas significantly higher than other learners in the first module. The public health nurses gave the best scores to the third module concerning the development of competence and the application of things learned being put into practice, and to the second module concerning the motivation for learning. Other learners scored the second module best in all satisfaction areas (Table 3).

The quantitative results were supplemented with qualitative results, which provided information about factors improving satisfaction within the course. The participants reported the e-learning course as filling the gaps in earlier education, having a clear and concise structure and content, an inspiring and interesting design and being suitable for clinical use (Figure 2).

The course was found to fill the gaps in earlier education. The interviewees felt that it improved their competence and strengthened their self-confidence. In particular, the case examples and exercises presented in the course were seen as supporting learning. The public health nurse students assessed the course as a suitable part of their studies and hoped it would be included as part of their degree education. The school nurses also felt that the course was suitable for continuing education; it was not too long to be carried out during the working day and offered new competences even to those who have worked as school nurses for many years:

According to interviewees, the course had a clear and concise structure and content. The course was considered technically easy to use and smooth running. It was felt that the content was clear and easy to absorb. The level of difficulty was also assessed as appropriate; there was both familiar and new information in the course. The course was deemed concise; there was nothing extra and nothing missing either. Splitting the course into three modules was a good and workable solution; the modules remained easy to perform:

The interviewees said the course had an inspiring and interesting design. The layout was considered vivid and visually delightful. The content and pedagogical choices were found to be pleasantly varied. Tasks encouraged reflection and thus supported deeper learning. The course also encouraged the participants to learn more; in particular, they desired to know more about the assessment rating scales and methods of psychosocial treatment. The participants interviewed were particularly satisfied with the variety of methods (text, pictures, video, audio and tasks) used in the course. Concerning the videos, their drawing format was especially liked. The videos were produced without sound using speech bubbles, which divided opinions:

The course was considered suitable for clinical use. The interviewees felt that the course was pragmatic; it contained practical tips and examples in the course that authentically reflected clinical situations. The participants were especially satisfied with the tools taught on the course, such as example phrases and action models. One school nurse reported that she had already adopted the five-meeting model taught in the course for managing anxiety. The course was also perceived as beneficial for pupils visiting school nurses; completing it was found to make it easier to discuss difficult topics and provide help immediately rather than refer pupils to other professionals. Many participants reported that they were going to return to the course later:

Our results show that the developed e-learning course was not only well participated and found satisfying by the target group but was also found beneficial by other health and social care professionals and students. Our findings provide new information about the factors related to participation and satisfaction, particularly when using Merrill’s (2002) principles to build e-learning courses. Despite the demonstrated benefits of using these principles (Badali et al., 2022), it appears that the principles have not been extensively used in the development of online health-care courses (Hendriks et al., 2020). Moreover, a recent review aimed at identifying theories supporting the design of digital learning interventions in nursing found a total of 20 different theories; however, Merrill’s model was not mentioned among them (O’Connor et al., 2023).

The high numbers of both public health nurse learners and other learners participating in the course indicate the demand for this kind of education not only in our target group but also on a wider scale. This finding is supported by previous studies showing the need for more competence in mental health issues, for example, among physicians (Spagnolo et al., 2018) and social workers (Kourgiantakis et al., 2022). The results may also indicate learners’ interest in the work practices of another professional group. Accordingly, one of the previously identified barriers to interprofessional collaboration is the lack of knowledge about each other’s work (Putkuri et al., 2023). An e-learning course such as the one examined can provide an easy way to observe another profession. Furthermore, one of the values of multiprofessional education is learning through the observation of others (Weil et al., 2018). The investigated course provided several opportunities for this, particularly through the video demonstrations.

Regarding participation within the course, the most interesting finding was that the integration of e-learning courses into postprocessing boosts diligence. This finding is similar to that for the flipped learning-method, where individual learning comes first and group learning after. Integration of e-learning into the wider context, for example through flipped learning, has been identified as one of the drivers of successful e-learning (Merrill, 2002; Badali et al., 2022; Regmi and Jones, 2020). Thus, for example, discussion in one’s own working community after completing an e-learning course about the things learned might be beneficial. Moreover, discussions within the work community about what has been learned could be a supportive way to use training targeted to another profession. Through discussion, it would be possible to consider what aspects of the training can be directly integrated into one’s own profession, what part needs adaptation and what part can be used in a way that provides information about the work of another profession, thereby supporting interprofessional collaboration.

The level of satisfaction was congruent between the target group and other learners. There was a statistically significant difference only concerning the 1st module (Prevention of mental health problems in school health care). Public health nurses scored this module higher than the other learners regarding the motivation for learning and the application of learned things in practice. This is understandable because of the preventive nature of the work of public health nurses. However, prevention should be given more attention in all health services (WHO, 2022). Hence, in the future it might be worth exploring the attitudes of other health professionals toward preventive mental health work. Regarding the development of competence, the public health nurses scored the third module (Depression, anxiety and suicidality in school health care) as being the best. This may indicate a need for more competence regarding treatment methods, because, despite treatment being excluded from school health services in Finland, school nurses also face pupils with mental disorders. Moreover, suicide is the most common cause of death among adolescents in Finland (SVT, 2023). The teaching of appropriate treatment methods may have been limited in degree programs. This interpretation is supported by the participants’ view of the course’s ability to fill gaps in previous education.

Our findings concerning factors related to satisfaction deepen the existing knowledge. Participants perceived the training as filling the gaps in their earlier education. Previous studies have already suggested that the degree training for school nurses does not provide sufficient expertise in mental health issues (Markkanen et al., 2021; Putkuri et al., 2021; Putkuri et al., 2023), and our results strengthen these findings. Participants in our study were satisfied with the clear and concise structure and content as well as with inspiring and interesting design. Previously, simple layout and clear navigation were identified as quality indicators of e-learning (de Leeuw et al., 2018). In our study, one of the key factors improving satisfaction was the suitability for clinical use. Practicality and authenticity are important factors in this regard, and their significance has been recognized in previous studies as well (Zaghab et al., 2015; de Leeuw et al., 2018; Merrill, 2002). Satisfaction was also increased by the ease of revisiting the material, which supports its use in clinical work. The significance of this has also been recognized previously (de Leeuw et al., 2018; Regmi and Jones, 2020). Furthermore, according to Merrill’s principles, good training provides not only information but also a demonstration of the subject being learned (Merrill, 2002). The significance of demonstration is also emphasized by Zaghab et al. (2015), and the scarcity of demonstrations in medical e-learning has been recognized by Hendriks et al. (2020). Participants in our study were especially satisfied with the practical situations demonstrated in the videos and the example phrases provided in the course.

In our study, we obtained new information regarding the suitability of Merrill’s first principles of Instruction method as a means of developing health sector e-learning and favorable results in terms of learner participation and satisfaction achieved through its implementation. Our results indicate that an e-learning course developed for a specific professional group has the potential for excellent adaption for several different professional groups.

This study has limitations and advantages. First, the course was designed for school nurses already active in working life, but despite many attempts, we only managed to obtain the participation of two of them in the qualitative part. Nevertheless, the participating students were close to graduation and had working experience in health care. Moreover, the questionnaire we used in the quantitative part did not allow for a distinction between public health nurses working in schools from those working elsewhere. Nevertheless, due to the subject of the course, it is assumed that many of the participating public health nurses were school nurses. Second, a self-selection bias may have influenced the quantitative sample favoring those more interested in the subject. Nonetheless, this large sample supports credibility. Third, there might be memory bias in qualitative results; some interviewees mentioned that some time had already passed since they completed the course. Nevertheless, the code saturation achieved after the first two interviews enhanced credibility. Finally, we only used a descriptive, posttest-only design, therefore, the results should be treated with caution and a pre-post-test study is needed in the future to assess the efficacy.

Tiina Putkuri has been employed by Duodecim Publishing Company Ltd., the copyright owner and publisher of the course during the development phase of the course. Anna S. Sarvasmaa has received funding from the company owning and publishing the e-learning course for participating in the development of the course. Other authors report no conflict of interest.

The developed course was evaluated as being well-participated and satisfying by the target group and other participants. The results demonstrate the demand for brief, fully online mental health training and indicate that learners perceive such training as beneficial. The findings provide evidence of the need to continue evaluation of the course with limited-efficacy testing to explore how well the next levels of learning outcomes (i.e. knowledge and competence) are reached.

Research Foundation of The Mannerheim League for Child Welfare (MLL). Zonta International District 20.

The authors thank Associate Professor Tuovi Hakulinen and President of the Finnish Association of Public Health Nurses (2018–2022) Tiina Mäenpää for developing the e-learning course. The authors also thank Heini Radi and other people in Oppiportti who helped with the research data, Miko Pasanen who provided consultation with statistical analysis and Elizabeth Nyman who provided language consultation.

The supplementary material for this article can be found online.