A study of the self-perceived competencies regarding the knowledge and skills of healthcare providers in the Expanded Program on Immunization in Thailand: A mixed-methods study Open Access

Immunization lies at the core of the control and prevention of major communicable diseases in Thailand. The provision of the Expanded Program on Immunization (EPI) in Thailand has received the praise of the World Health Organization as one of the 13 countries worldwide to have met with success in developing a vaccine-preventable disease-monitoring system for child immunization[1]. The policy has required that from birth, Thai children receive primary immunization against 11 diseases, aimed at the ultimate eradication and elimination of vaccine-preventable communicable disease[2]. In order to achieve its main objective, the successful development of EPI in Thailand has depended on the following four components: the vaccine component, produced in accordance with set standards; the budgetary component, which supports the production and research of new vaccines that will stay abreast of newly occurring diseases; the administrative component that manages production, storage and delivery and management within the service units; and personnel or providers engaged in immunization, which is a very important component inasmuch as it functions as the main mechanism and lies at the very heart of the process that will steer the EPI’s aims toward achieving its objective[3].

It is currently estimated that there are approximately 20,000 EPI staff members in Thailand. However, it was discovered that there are regular personnel transfers in this line of work, causing the services being provided in the area of immunization to frequently fluctuate. Moreover, the nature of this line of work requires the development of continually updated knowledge and skills in rendering services, since the specialized knowledge needed in the administration of vaccines continually undergoes change, and new concepts are constantly arising. For this reason, staff engaged in the work of immunization are crucial to the mechanism that will determine whether the process of immunization in Thailand will achieve the desired work standard. A review of the service standards pertinent to the work of immunization found that carrying out this work in Thailand still had problems of several kinds. Based on information gleaned from the supervision of work performed in the area of immunization by the Department of Disease Control, issues were still found pertaining to services that did not comply with standards, such as the vaccine management system and the cold chain system, which continued to be non-standard compliant[4]. The aforementioned data were either a reflection of human error, incorrect understanding or low skills. Therefore, it is crucial that the managers of the immunization program, especially in the provider group in Thailand, be at least minimally aware of knowledge-and-skill competency levels and how immunization services are provided.

To date, numerous studies have been carried out on parental attitudes toward immunization and healthcare providers[5–7], the vaccination competency of nurse students and nurses[8–10] and knowledge or vaccination competency from the perspective of immunization providers and clients overseas, especially in western countries[11–13]. However, there is no available research on the knowledge and skills of those providing immunization services in Thailand. Consequently, this research describes a systematic study that will provide accurate data on the knowledge-and-skill competency awareness of service providers. These data were gathered in six regions of Thailand to provide an overall picture nationwide. Research findings can serve as empirical evidence for policy setting, and for a plan of work or for guidelines for furthering the capabilities of personnel on a level consistent with the expected competency of a provider. This information can then be applied to the development of work in a continuous and systematic fashion, and contribute to bringing the work of immunization to a greater level of success than before.

This study used a mixed-methods approach using a triangulation convergence model. For the convergence model, the researchers collected and analyzed quantitative and qualitative data separately and then the quantitative and qualitative findings were converged by comparing and contrasting findings during the interpretation[14].

In the quantitative method phase, participants were EPI staff consisting of physicians, nurses, pharmacists, public health scholars and public health officials. The sample size was determined using the G*power program 3.1.9.2. software. A sample size of 272 persons was required to achieve a power of 0.95 and an effect size of 0.2 at the α level of 0.05. Because the response rate in the postal mail survey was usually low, the sample number was increased by 50 percent. Therefore, the sample size that was expected to be used contained a total of 408 persons, requiring data to be compiled using the stratified random sampling method. Data were taken from those performing the work in six regions nationwide. One province was then selected from each region: Chon Buri in the East, Phetchaburi in the West, Chiang Mai in the North, Krabi in the South, Khon Kaen in the northeast and Bangkok in the central region.

For the qualitative methods phase of this study, four healthcare providers involved in immunization service from one health service setting in each region were selected through purposive sampling for in-depth interviews and non-participant observation. In summary, six health service settings were included in this phase and 24 healthcare providers from the six regions were interviewed and observed while performing their EPI services.

Two research instruments were employed: the demographic questionnaire and the Perceived Self-Competency for EPI Service Provision Evaluation Form for the quantitative phase of the study. Guidelines for the in-depth interviews and for the observations were used in the qualitative phase.

A demographic questionnaire was used to obtain personal data on the EPI staff, including sex, age, education level, length of time working in the field of immunization and characteristics of work.

The Perceived Self-Competency for EPI Service Provision Evaluation Form was developed by the researchers for measuring the staff member’s perception of his/her competency in performing the work of immunization, including the self-confidence of the person in his/her knowledge and skill in rendering the service and managing the work of immunization. This tool contained a total of 80 questions covering the competency of EPI service based on the standard of service for the EPI training program of the National Vaccine Institute and the quality standards for immunization practices in the Department of Disease Control, Thai Ministry of Public Health. This instrument rated each item by using a five-point Likert scale that ranged from 1 (indicating no confidence) to 5 (indicating high confidence). The score results were subdivided into three levels based on criteria derived from Best’s criteria[15]. These were 1.00–2.33, which meant a low perception of competency; 2.34–3.67, which meant a moderate perception of competency; and 3.68–5.00, which meant a high perception of competency. Content validity was assessed by five experts. The Index of Item Objective Congruence (IOC) was 0.98. The result of the internal consistency using the Cronbach’s α was 0.97.

In-depth interview guidelines for the EPI staff were used pertaining to the perceptions of competency in EPI service. The interview guidelines included five items: How do you usually provide immunization services?; As an EPI staff member, how confident are you in providing immunization services?; Which performance do you have high-perceived competence in and which do you not?; What kind of difficulties have you encountered? Why?; and What are your expectations regarding the development of your knowledge and skills in providing immunization services? Instrument quality was assessed for content validity by five experts and tested by two EPI staff members in order to ascertain their understanding of the questions.

The Evaluation for EPI Service Observation Guidelines was used to assess the work being performed in a well-baby clinic. The guidelines contained a 51-item checklist that needed to be followed and monitored for the EPI service and was developed based on the evaluation forms for the quality standards for immunization practices, Department of Disease Control. Content validity was assessed by five experts and reliability was high (IOC=0.91, inter-rater reliability=0.90). The skills in providing immunization service and all activities encountered by an EPI staff member were also collected through field notes that the researchers took during observation. Field notes were written while the researchers were observing.

For the quantitative method phase, the questionnaires, consisting of the demographic questionnaire and the Perceived Self-Competency for EPI Service Provision Evaluation Form, were sent out to approximately 70 EPI staff members in each provinces in the six regions – Chon Buri, Phetchaburi, Chiang Mai, Krabi, Khon Kaen and Bangkok – by postal mail, and 91 percent of them – 382 persons – returned their questionnaires.

For the qualitative method phase, the researchers selected the participants from an EPI service unit in each of the randomly chosen provinces as they did with the quantitative data. Four EPI staff members in the selected service unit were purposively recruited as key informants per unit. The purpose of this phase was to conduct interviews and non-participant observation on the work being performed in the area of immunization by using the in-depth interview guideline for the EPI staff and the observation guideline. During observation, the researchers took field notes to record the activities undertaken that may not have been identified from the audio recording.

Data from the quantitative method were analyzed by using SPSS version 22 with the level of significance at 0.05 in order to describe the frequency and percentage of the demographic data and the perceptions of competency in the administration of immunization of the EPI staff. The data derived from the qualitative method consisted of data based on behavioral observations for which frequencies and percentages were calculated. Additionally, the data from the interviews and field notes from each observation were analyzed using content analysis[16]. The qualitative data analysis was joined with the quantitative data (triangulation) by comparing the two data types in order to arrive at the research results.

This research was approved by the Second Ethics Subcommittee on Human Research, Thammasat University, No. 024/2558. The information sheet that explained the data were sent to those that had joined the research project and to everyone in the sample groups in order to explain the details of the project and the protection of rights that applied to the sample groups. All participants who agreed to be a part of this study were required to sign the consent form that was then returned by postal mail together with the questionnaire.

Regarding the quantitative method phase, among the total of 382 participants, 320 were female (83.8 percent) and 62 were male (16.2 percent), with an average age of 40.8±9.04 years. Results showed that 21.5 percent of the participants were between the ages of 36 and 40 years, followed by 18.3 percent that were within 41–45 years of age. Furthermore, 77 percent of the participants had finished their education at bachelor’s degree level. A 68.3 percent majority of the participants were working in the nursing profession, followed by an estimated 19.1 percent of individuals who were public health staff members. The average length of time spent working in the area of immunization was 7.3 years (SD=6.8), with 43.7 percent of the participants having worked from one to five years, followed by 23.0 percent that worked from 5.1 to 10 years. As regards the workplaces, 48.5 percent of the participants worked in primary healthcare centers or in health-promoting hospitals in sub-districts, followed by 26.4 percent that worked in community hospitals. Additionally, 70.4 percent of the participants had been previously trained on an immunization course.

The results for the quantitative data showed that the participants had a high perception of knowledge-and-skill competency in EPI service. They had an average score of 3.95 (SD=0.61). When compared to the competency score as classified by EPI service, it was found that the participants had the highest perceptions of competency in preparing and administering the vaccines (⁠$x¯=4.12$⁠, SD=0.69), followed by vaccine storage and cold chain system (⁠$x¯=4.05$⁠, SD=0.71), and the lowest perceptions of competency in dealing with adverse events following immunization (⁠$x¯=3.82$⁠). The details are as shown in Table I.

The results for non-participant observations of the performance of EPI services showed that what the participants did was either incomplete or incorrect according to standards for administering the vaccines, vaccine storage, vaccine estimation and vaccine report registration, and dealing with adverse events following immunization. Details are shown in Table II.

The qualitative findings of both observational and interview data revealed that the participants were confident in their skills in administering the vaccine shots. They also felt that they were proficient in their work and in developing work standards in the areas of vaccine storage based on the cold chain system. On the other hand, participants were not confident in their performance of the following: scheduling immunization appointments for a group that was afflicted with health problems or that had come in for vaccinations later than the time appointed; vaccine estimation and vaccine report registration; and caring for adverse events following immunization.

Both quantitative and qualitative findings were used to summarize the knowledge-and-skill competency of the EPI staff members in Thailand. This summary proceeded as indicated below.

Category 1: those involved in the work had high competency perceptions. The work they performed conformed to standards of preparation of the site and equipment for vaccine services and in reporting to parents on giving vaccines. The details are shown in Table III.

Category 2: while those doing the work had high competency perceptions, the work they did was either incomplete or incorrect according to standards for providing services and for vaccine storage. The details are shown in Table IV.

Category 3: although the participants had a high perception of their knowledge-and-skill competency in all aspects of EPI service, they had the lowest of the three levels of perceived competency as follows: administering basic life support, and referring the patient when adverse reactions occurred; identifying the severity of symptoms following immunization and reporting the coverage for routine vaccination services. Their work was either incomplete or incorrect according to standards for making vaccine estimations, registering reports and dealing with adverse events following immunization (Table V).

According to the results of the study that examined the perception levels of knowledge-and-skill competency in EPI service, it was found that the participants had high competency perception levels in EPI service provision (⁠$x¯=3.95$⁠, SD=0.61). The reason may be that a 70.4 percent majority of the participants had previously received training in immunization, so they had gained practical knowledge in the administration of vaccinations from experts or from those that were qualified in this field, and then they passed on their own expertise. Nowadays, training programs – three-day programs and one-day programs – are usually completed annually by the Thailand National Vaccine Institute and Department of Disease Control. Many printed materials such as immunization handbooks, guidelines and textbooks are distributed throughout the country both in hard copy format and electronically uploaded files. The monitoring system in each Area Health is randomly carried out by the Division of Vaccine Preventable Disease, Department of Disease Control, on an annual basis. The participants thus acquired knowledge and experience from others and eventually developed a perception of their own competency[17, 18]. This corroborates a theory of Bandura[19], which offers a clear explanation of participant results, namely, that they had already been trained in immunization. The results indicate that they benefited significantly from good role models that had provided them with their knowledge and shared important experiences. Their output consisted of developing the knowledge of service providers and reducing problems and obstacles in work supervision through access to resources for counseling in the performance of their work. This generated clarity in the advice being offered and built up awareness of their job performance[20]. The fact that the participants had a high level of perception of competency in rendering their services in immunization may have resulted from their having worked for long periods of time, an average of seven years, and had an extensive amount of work experience. Furthermore, their experience was directly in the area of immunization, and they were bound to develop self-competency awareness to a high level. It was further revealed that the participants had the highest perceptions of competency in preparing and administering the vaccines, vaccine storage and use of the cold chain system. The above findings concur with a statement of Bandura, who asserted that performing a task successfully strengthens one’s sense of self-efficacy, thus leading to successfully accomplishing a task. In other words, having direct experience of mastering something is a powerful way of increasing one’s self-efficacy, and in this case, influenced the initiation and maintenance of vaccination competence[19].

Meanwhile, the participants had moderate and low perceptions of competency in the areas of administering care whenever adverse reactions following immunization occurred, and in making vaccine estimations and registering reports. The reason may be that the participants consisted mostly of nurses that had no specific duty or direct responsibility in these tasks. The very fact of never having had any experience in carrying out these functions or any specific duty or direct responsibility in carrying them out tends not to generate any perception of competency in work performance, and the work itself may be incomplete or non-standard. Yet, at the same time, there were participants with perceptions of competency that were high, but whose work was either incomplete or noncompliant with standards, especially in administering the vaccine shots and in their storage of the vaccines. There were other factors that affected their personal actions, including, in particular, environmental factors, or the context of the individual[21]. These factors led to job performance in which the results were not correct. For example, the arrangement of the vaccines in the refrigeration units was incorrect because of the limitation of the refrigeration equipment. There was no recording of the refrigeration temperature on the weekends from morning till evening, nor had any instruments been prepared to provide first-aid in the event of undesirable symptoms following immunization. Neither was a suitable space prepared for the observation of post-vaccination symptoms. This situation was the result of the work policies of the immunization units, which were very diverse. They depended on the context of the service sites and the affiliated principal work units. Accordingly, the immunization work systems were implemented in ways that generally differed. They may not have harmonized with the evaluation forms for the standards of the primary vaccine development group, Department of Disease Control, which served as the template in the construction of the research tools for the present study. Furthermore, the policy limitations within each of the work units caused the failure of budgetary support for the work unit itself and led to a shortage of personnel as well. These findings were consistent in that the nurses that were administering immunization shots were knowledgeable in their work, but were subject to certain limitations. Among these limitations was a lack of staff in the workplace, which impeded the services they were providing in administering the vaccines[22].

The data from this research indicated that the individuals engaged in this work have high-level perceptions of their own competency, which serves as a reflection of their knowledge and skill in immunization practices. Nonetheless, some problematic issues were found in the work of the healthcare providers that conflicted with certain stipulated work standards. Accordingly, these findings serve as clear evidence of the need to develop the competency of EPI staff in order to overcome the barriers and to sustain effective services in Thailand.

This study provides insights into national organizations in Thailand including the Thailand National Vaccine Institute or the Division of Vaccine Preventable Disease and the Department of Disease Control. The study can support recommendations for planning staff development policy to enhance adequate and systematic training, monitoring and evaluation throughout the EPI staff’s working life. Moreover, the effectiveness of traditional training might be reviewed and added to new methods that will be able to overcome the constraints between high-perceived competence and unmet standard practices. Additionally, scheduling and rescheduling of immunization, vaccine storage in the cold chain system, vaccine administration and preparation for adverse effects following immunization should be more closely monitored.

The researchers wish to thank the participants for providing useful information and also the National Vaccine Institute for providing the funds for this study.