Evaluating the Effectiveness of Online Faculty Development in Creating Accessible Content Free

Online and Web-assisted learning has expand-ed the reach of educational programs to reach previously underserved learners. The trend to-wards online education has broadened educa-tional access. Despite this, learners with phys-ical or intellectual disabilities may encounter barriers that impede their learning. “Millions of individuals in the United States have dis-abilities that affect their use of the Web. Many of these individuals use assistive technology to enable them to navigate websites or access information contained on those sites” (US Department of Justice, 2012, n.p.). For instance, Rose and Gravel (Cited in Gordon, Gravel,&Schifter, 2019) discuss, “Learners with disabilities are most vulnerable to such barriers, but many students without disabilities also find that curricula are poorly designed to meet their learning needs” (p. 5).

Designing accessible content is an import-ant step in ensuring that web-based learning materials are accessible to all users. Specific concerns related to ensuring accessibility of web-based learning materials include the provision of captions for audiovisual resources to ensure equal access for deaf and hard-of-hearing populations. The use of proper heading layouts and image descriptions for individuals who have vision problems, ensuring that color and contrast are appropriate, so items are legible, and promoting universal design for all learners.

Designing accessible content provides ben-efits for diverse student populations, as stu-dents can interact with learning materials in a variety of ways. For example, closed captions for a video are primarily intended to help deaf or hard-of-hearing learners, captions can also be beneficial for other students. For instance, captioned videos allow for English language learners to understand what is being presented, which may be beneficial to those who are working in noisy settings.

The provision of accessible content in on-line learning environments helps to improve the teaching and learning process for students with disabilities enrolled in online courses. For instance, Hashey and Stahl (2014) explain: “The decisions educators make regarding on-line instructional resources are perhaps more critical to students’ success than decisions about print-based materials because learning occurs exclusively through and within this environment” (p. 71). Therefore, accessibility should be at the forefront of instructors when developing online courses. However, “This ease of access is simply not the case for many web-based tools and content” (Hashey&Stahl, 2014, p. 71). This example emphasizes that instructors should be skilled in identifying and assessing the accessibility of online con-tent in order to produce a learning environment that is beneficial to all users.

The provision of accessible content rep-resents a legal obligation. According to leg-islation such as the Americans with Disabilities Act (ADA), institutions who receive federal funding must ensure that all students with disabilities must have equal access to all learning materials in a class. Failure to provide accessible materials could lead to fines and lawsuits being levied against the institution. For instance, in 2015, the University of California at Berkeley was determined by the US Department of Justice to have violated the ADA. “Berkeley’s free online educational con-tent was inaccessible to blind and deaf people because of a lack of captions, screen reader compatibility and other issues” (Straumshein, 2017, n.p.).

A related issue for institutions is balancing their need to provide accessible content while maintaining fiscal prudence. When the Univer-sity of California at Berkeley was sued by the Department of Justice for hosting inaccessible content, the institution was required to change how they had made content available to the public:

The actions taken by the University of Cali-fornia at Berkeley involved removing public access to content that was inaccessible. The university worked to create a new content repository with accessible content which re-quired users to register and authenticate in order to access the content. (Koshland, n.p, 2017).

A proactive approach to learning envi-ronment design to accommodate all learners could resolve issues with content inacces-sibility. The theory of Universal Design for Learning, or UDL, is used to design learning environments that are universally accessible to users with disabilities and meets the needs of diverse learners. UDL emerged from scientific research conducted by the Center for Applied Special Technology (CAST). According to CAST (2010), “UDL is an approach that min-imizes barriers and maximizes learning for all students” (0:43).

At a large Midwestern research university, a faculty professional development session was developed by the instructional design staff to instruct faculty on how to create accessible web content. The developmental session covers topics related to legal compliance, the basics of UDL, and skills required to develop accessible course content.

Creating web-accessible learning materials is a requirement that many faculty members are unaware of, and many faculty members do not have the technical knowledge required to create accessible learning materials. Regarding this issue, Vollenwyder et al. (2019) discuss: “A further challenge is the formation of ade-quate knowledge and skill for the actual implementation by all involved web practitioners. Effective Web Accessibility requires a thorough understanding of how design and implementation of a solution can address the needs of users with disabilities” (p. 353).

Students with disabilities represent a sig-nificant demographic in education that is frequently underserved by educational insti-tutions. According to the World Health Orga-nization (WHO) (2011), “There are over one billion people with disabilities in the world, of whom between 110–190 million experience very significant difficulties. This corresponds to about 15% of the world’s population” (p. 1).

In addition, research pertaining to Universal Design for Learning (UDL) is limited, and this study will help to expand knowledge on the subject. Scott, Temple, and Marshall (2015) discuss, “Despite the fact that UDL can be a strong tool in coursework design, there are few studies which have examined the specific use of UDL principles in post-secondary environments, and fewer studies which have characterized by true experimental designs” (p. 102).

The issue of ensuring accessibility and equality in education can be a difficult prop-osition. For example, Gordon, et al. (2009), state, “The law on the books calls for access to the general curriculum for students regardless of ability or disability, but realizing that vision requires more than simply telling people what the law requires” (p. xii). Therefore, while fac-ulty, administrators, and public policy makers may be familiar with accessibility laws, they may struggle with developing a vision for its implementation.

This research study explored the efficacy of providing online professional development for faculty in developing accessible online course materials. This study evaluated faculty atti-tudes, beliefs, skills, and abilities related to the creating of accessible online course content.

The population of this study was faculty members who taught online courses using Blackboard Learn at a large midwestern regional research university. Inclusion criteria included the use of Blackboard Learn for hosting course materials. Faculty members who did not use Blackboard Learn, or who used educational materials hosted on sites outside of Blackboard Learn were excluded from the study.

Two research groups were established for this study. The first group consisted of faculty who had completed online professional devel-opment related to building accessible online courses. Since the number of faculty members who had completed the course was a small portion of the total faculty, research invitations were sent to all participants of the course. The second group included randomly selected faculty from the population who had not participated in professional development related to Web Accessibility.

1. Is there a difference amongst faculty attitudes towards accessible content for those who have completed web accessi-bility professional development in con-trast to those who have not?

2. Is there a difference amongst faculty skills in creating accessible content amongst faculty who have completed web accessibility professional develop-ment in contrast to those who have not?

3. Is there a difference in the level of web accessibility of courses taught by faculty who have completed Web accessibility professional development.in contrast to those who have not?

4. Does faculty awareness of Universal Design for Learning (UDL) have an im-pact on the level of web accessibility of courses taught by faculty?

A survey instrument was distributed to the se-lected faculty members. The survey instrument included questions related to general technical ability, awareness of accessibility requirements, and specific skills related to creating accessible web-based course materials. The survey instrument included three parts.

The first part consisted of 24 items adapted from the survey instrument developed by Vol- lenwyder, et al. (2019) regarding attitudes and behavioral predictors towards Web Accessibil-ity. Vollenwyder, et al. (2019) outlined a series of salient beliefs related to Web Accessibility that were examined such as personal effort, social responsibility, business opportunity,product quality, user advocacy, legal obliga-tions, self-perception as specialist, awareness and priorities, requirement conflicts, technical compatibility, limited resources, and knowl-edge and skills. These beliefs were then cat-egorized as attitudes, subjective norms, per-ceived control, actual control and background factors. (p. 354). Users were asked to indicate their agreement with each item on a four-point scale, with 1 indicating “strongly disagree” and 4 indicating “strongly agree”. The questions were adapted to incorporate language that is more suitable for instructors at the studied institution.

The second part of the survey instrument asked users to evaluate their level of aware-ness regarding accessibility policies. This part of the instrument was based on questions de-veloped by Mancilla and Frey (2021). Such questions included awareness about student support services, institutional accessibility policies, and support for implementing acces-sible course design.

The third part of the survey instrument asked users to evaluate their capability with creating accessible course technologies, as well as identifying technologies that present challenges for them, and areas that users would indicate additional training needs.

Demographic information included in the survey instrument included the faculty status of the respondent, and how many semesters they had taught online. Respondents are also asked if they had used the Blackboard Ally tools within their course.

Faculty members were asked if they were familiar with the concepts related to UDL, and the accessibility levels of their course materials were evaluated using Blackboard Ally. The accessibility of individual courses taught by faculty were evaluated using the Blackboard Ally dashboard scores within the LMS. Overall accessibility was scored on a scale of zero to 100, with a score of zero indicating a lowlevel of accessibility, and 100 indicating a high level of accessibility.

The questionnaire was distributed to randomly selected faculty members amongst the popu-lation, as well as to faculty members who had completed the ADA Compliance and Online Courses faculty development course. The questionnaire was sent to 74 participants of the professional development course, and to 216 randomly selected faculty members with-in the total population. The total number of respondents was 290. Of the 290 invitations sent, 31 individuals accessed the survey, and 23 individuals completed the survey. The re-sponse rate was 10.6%, with a completion rate of 74.1%. Survey invitations were distributed using the Qualtrics XM platform, with periodic reminders sent out over five weeks.

The questionnaire was distributed during the height of the COVID-19 pandemic, which forced many social, educational, and research activities into digital spaces. Consequently, the uptake of digital events and interactions led to burnout and survey fatigue amongst the popu-lation. For example, de Koning, et al. (2021) acknowledged the increased survey distribu-tion during the COVID-19 pandemic “Has led to potential survey respondents being ap-proached more frequently within a short peri-od, leading to a type of survey fatigue in which these respondents refuse to complete surveys at all” (p. 2).

Questionnaire items used for Research Question One included both scaled and dichot-omous items. The items measured different aspects related to attitudes and beliefs regarding the development of accessible online environments. In addition, users were asked to identify various accessibility policy positions. A Mann-Whitney U test was performed to compare group performance on scaled items, while a Chi-Square test was performed to compare group performance on dichotomous items.

A Mann-Whitney U statistical test is ap-propriate for comparing distributions between two group of ordinal dependent variables. For dichotomous items, a Chi-Square test is ap-propriate for comparing group performance against binary dependent variables. (Laerd Statistics, 2015).

Questionnaire items for Research Question Two included scaled, multiple-response, and dichotomous items. The multiple-response items were scored as dichotomous item pairs. These items measured individual faculty member’s skills and understanding of twelve key skills for designing accessible courses. Participants were asked to identify which specific practices they utilized in their courses, which included descriptive hyperlinks, al-ternative text, alternative formats (e.g. audio, video, text, images), headings, readable PDF files, table design, captioning/transcripts doc-ument design, font colors and contrasts, plain language (e.g. familiar language, active voice, concise sentences), keyboard accessibility, and consistent navigation menus.

Participants were asked to identify the level of effort to carry out the specified accessibility practices, along with which items pose the most challenge, and areas in which instructors felt they needed additional training. Dichotomous items were scored using a Chi-Squared statistical test to identify differences between the two study groups, while a Mann-Whitney U statistical test was used to calculate differences in performance between the two study groups on the scaled questionnaire items. A Mann-Whitney U statistical test is appropriate for comparing distributions between two group of ordinal dependent variables. For dichotomous items, a Chi-Square test is appropriate for comparing group performance against binary dependent variables. (Laerd Statistics, 2015).

Research Question Three included a con-tinuous dependent variable (Blackboard Ally Accessibility Scores), and a categorical inde-pendent variable (faculty participation in the ADA Online Accessibility course). The two groups identified were independent, i.e., a single member would not be included in both the experimental group and the control group. Since this research question utilized a con-tinuous dependent variable, and a categorical independent variable, an Analysis of Variance (ANOVA) statistical test was identified as an appropriate statistical test to use (Laerd Statis-tics, 2015).

Research Question Four included a con-tinuous dependent variable (Blackboard Ally Accessibility Scores) and a categorical inde-pendent variable (faculty self-identification of being familiar with UDL principles). The two groups identified were independent of obser-vation. Since this research question utilized a continuous dependent variable and a categori-cal independent variable, an Analysis of Vari-ance (ANOVA) statistical test was identified as an appropriate statistical test to use (Laerd Statistics, 2015).

The research participants were asked to de-scribe the following demographic identities: Faculty status, number of semesters taught online, completion of the ADA Compliance and Online Courses professional development course, usage of the Blackboard Ally accessi-bility tool, and familiarity with UDL. Partici-pants were also asked to provide the number of semesters they had taught online. Other de-mographic information collected from partici-pants included whether they had participated in the ADA Compliance and Online Courses professional development course, whether fac-ulty have used the Blackboard Ally to improve accessibility of course material, and whether faculty were familiar with the principles of UDL.

This study utilized a 43-item questionnaire, as well as course-level data from Blackboard Ally for courses taught between Spring 2020 and Spring 2022. The questionnaire was sent to 290 individual faculty members, and 31 faculty members responded to the survey, for a response rate of 10.6%. The study population included faculty members at a Midwestern re-gional research university.

Research Question One. Research Ques-tion One was: “Is there a difference amongst faculty attitudes towards accessible content for those who have completed web accessi-bility training in contrast to those who have not?” This research question explored various attitudes related to developing accessible web-based learning environments, as well as the identification of institutional policies and practices related to web accessibility. Themes related to this research question were explored in part one of the questionnaire, which was based on research conducted by Vollenyder, et al. (2017). The research included the following beliefs related to the consideration of Web Ac-cessibility: Personal effort, social responsibili-ty, product quality, user advocacy, legal obligations, self-perception as specialist, awareness and priorities, requirement conflicts, technical compatibility, resources, and knowledge and skills (p. 354).

From the participants’ responses, there were no findings of significance related to these identified beliefs. Most research participants indicated that they had a positive view towards creating accessible content. Participants also showed a willingness to consider the incorporation of accessible web content for future courses they plan to teach. This willingness stems from users anticipating students and peers would view accessible web course resources positively. Additionally, most faculty members agreed that the consideration of Web Accessibility (WA) was an institutional priority.

However, some faculty members indicated that a lack of technical skills or access to time and resources necessary to consider WA in course development. For users who completed the professional development, this finding indicates a gap between skills taught in the professional development course and what participants had reported in the questionnaire. A further needs analysis related to desired pro-fessional development and other necessary resources should be performed to determine additional resources that may be helpful to faculty in helping them to overcome limitations with a perceived lack of resources and skills for developing accessible course materials.

Professional development related to creat-ing accessible course environments should fo-cus on developing faculty skills for designing accessible Web environments. Professional development should then be focused on low-ering the learning curve necessary for creating accessible content. Regarding professional de-velopment, Vollenwyder, et al. (2019) discuss-es, “Web practitioners’ knowledge and skill of how to effectively work on Web Accessibility should be continuously supported, because it benefits their self-perception as specialists” (p. 358). Therefore, it is important for educational institutions to support the professional development of faculty members by providing resources, tools, and instruction in developing accessible course materials.

While the research study did not show a sig-nificant difference between those who took the ADA Compliance and Online Courses professional development course for most items, the study did point to some deficiencies in policy awareness that could be addressed through improvements in communication and professional development. The study indicated that the participants in the sample likely had awareness of accessibility requirements. Participants also expressed a desire to make course materials accessible, but some participants acknowledged a deficit of skills for making accessible content, as well as feeling they lacked the necessary resources for creating accessible content.

Research Question Two. Research Ques-tion Two explored whether participation in professional development resulted in users be-ing more proficient with a selection of skills related to the creation of accessible content. The questionnaire were used to explore topics related to transcription and captioning of au-diovisual media, usage of selected accessible contents, and skills related to creating such content.

User performance on these items is noted as follows:

1. Based on the data collected, and the sta-tistical tests performed, no significant difference between the two groups of faculty members was identified. However, faculty reported a variety of frequencies for providing transcripts for media.

2. Of the faculty members who responded, 13 (or 52%) faculty members indicated that course audiovisual materials were captioned “always” or “often,” while 12 faculty members reported that audiovisual materials were captioned “sometimes,” “rarely,” or “never.” This value falls below the percentage of respondents who reported that captions were provided “always” or “often” of 61% on the study by Mancilla and Frey (2021).

3. The use of descriptive hyperlinks and alternative text were the two most-utilized skills, while accessible document design and accessible table design were the least utilized. Meanwhile, headings, accessible document design, alternative formats, and consistent navigation represented a medium level of effort to implement.

4. Faculty members indicated that document design, headings, and table design posed a medium challenge, while plain language and font colors and contrasts represent the items that posed the lowest challenge.

5. The participating faculty members reported that alternative formats represented the area they felt needed more training in creating readable PDFs, accessible document design, and keyboard accessi-bility.

6. Instructors felt they needed additional training for items such as alternative formats and creation of captions and transcripts for multimedia objects repre-sented the items instructors felt were the most difficult to implement, as well as the areas that faculty members felt they needed additional training for. The creation of alternative formats and captioning of audiovisual materials represented the accessible design elements for which participants of Mancilla and Frey (2021) had indicated were the most difficult to implement.

Research Question Three. Research Ques-tion Three was: Is there a difference between the level of accessibility of courses taught by faculty who completed the ADA Compliance and Online Courses professional development course compared to those who did not. Course accessibility was measured using overall Blackboard Ally accessibility scores, and a mean score was calculated for each participant. Overall, a significant difference was identified between mean accessibility scores for the two groups of faculty members. Faculty members who had completed training had higher mean accessibility scores. The value for significance was p = 0.023 (p<0.05). Faculty members who have completed professional development related to creating accessible course materials were more likely to have higher levels of accessibility in their courses.

Considering the importance of developing accessible course materials from both a legal and a usability standpoint, professional development related to creating accessible course materials should be offered to faculty on a broad basis. As more faculty members take part in professional development related to creating accessible online courses, the over-all accessibility of online courses can be im-proved on an institutional level.

The literature review showed that the avail-ability of accessibility evaluation tools such as WebAIM or Blackboard Ally has led to more faculty and instructional developers using these tools to improve the accessibility of online learning environments. Professional de-velopment should be provided to faculty members on effective use of accessibility evaluation tools and the process of accessible content creation in software programs that are frequently used by faculty members. (Mancilla&Frey, 2020). While tools such as Ally can help to improve Web Accessibility, there is concern that such tools can create a misleading picture of course accessibility (Lieberman, 2018).

Research Question Four. Research Ques-tion Four asked if “Awareness of UDL prin-ciples contributed to higher levels of course accessibility compared to faculty members who did not express awareness of UDL”. In the questionnaire, faculty members were asked to self-report if they were familiar with the principles of UDL. Of the faculty members who participated in the questionnaire, 20 fac-ulty members reported they were familiar with UDL principles, while six faculty members did not. There was no significant difference in ac-cessibility scores for these two groups of fac-ulty members.

It is likely that the faculty surveyed had a high understanding of UDL concepts, even if they did not partake in professional devel-opment related to accessibility. Ultimately, a thorough understanding and familiarity of UDL principles among faculty at the institu-tional level can help to advocate for improved accessibility of online courses, and to promote more equitable learning experiences for stu-dents.

Existing studies may indicate that faculty members at different institutions may have familiarity with at least one principle related to UDL. In turn, faculty members who do express familiarity are often willing to apply principles of UDL within the courses they teach (Westline, et al., 2019; Scott, Temple&Marshall, 2015).

The growth of the Internet as a medium for instruction over the past few decades, in the form of online learning, web-assisted courses, and blended learning has created many oppor-tunities and challenges for expanding educa-tional access to diverse learners. This trend, as well as the switch to remote learning options during the global COVID-19 pandemic, also shows that accessibility of web-based instruc-tional content is an important consideration for instructional best practices and legal compliance.

This research study explored faculty atti-tudes and beliefs related to the creation of ac-cessible content and the implementation of best practices for Web Accessibility. Faculty members understood the legal concepts related to accessibility and disability law. Participants also expressed positive desires for creating accessible content as well as a willingness to meet student expectations of accessible content.

Faculty members expressed that they did not always have the appropriate knowledge and skills or the resources to make content ac-cessible in their courses. This belief was ex-pressed consistently among faculty who com-pleted professional development, as well as those who did not. Therefore, additional focus should be placed on developing faculty skills and confidence with developing accessible course materials.

The research study showed faculty users who received professional development in un-derstanding accessible course design were able to apply that knowledge to improve accessibil-ity in their courses. Tools such as Blackboard Ally allowed faculty members to supplement professional development by providing feed-back for how instructors can improve their course materials, as well as providing students with alternative formats that can benefit stu-dent learning.

The use of Blackboard Ally helps to realize the potential for UDL in online course design by helping to adapt the learning experience for students with different learning needs. Scott, et al., (2015) mentions, “UDL as a framework for online coursework and preparation might ensure a quality learning experience for students. It may also help teacher preparation programs seeking to design and deliver quality instructional experiences for students, and help college programs maintain a level of quality that will improve online teacher preparation” (p. 100).

It is important for institutions to support accessibility and inclusion at the institutional level. Mancilla and Frey (2021) demonstrated the importance of institutional support for pro-moting course accessibility. “It is critical for campus administrators to establish a culture of inclusivity that undergirds all online course development efforts and prioritizes the digital accessibility of instructional materials” (p. 11). Therefore, institutions should encourage the creation of accessible online course materials by investing in resources to facilitate the cre-ation of accessible online course materials, as well as providing professional development and support to course instructors, instructional designers, and others involved in the content creation process.

The findings of this research study indicate, that amongst the faculty members surveyed, a desire to create accessible web content for the purposes of ensuring legal compliance as well as satisfying the need to accommodate students with different learning styles and abil-ities. While there was not a significant differ-ence between faculty members who participated in training in contrast to those who did not participate in training, this study indicates that there is value in developing positive faculty attitudes towards accessibility.

While the study did not find a significant difference in skill level amongst faculty members who participated in professional development, versus those who did not, there was a significant difference between the level of accessibility of course content. Finally, the principles of UDL were broadly understood by the surveyed faculty, understanding of UDL did not have a significant effect on the level of accessibility of course content. However, UDL is a key component of creating accessible and engaging online learning experiences for students.