Accessible Online Learning: An Analysis of Online Quality Assurance Frameworks

Accessibility is a popular topic in online education today (Burgstahler, 2015; Lederman, 2017; Lee, 2017; Moorefield-Lang et al., 2016). Despite the increased attention, though, most discussions about creating accessible online courses focus on the basics, such as the need to add alternative text descriptions to images and captions to videos (Silver, 2016). While research has shown that strategies that make content accessible for students with disabilities (e.g., the use of captions) are helpful for all learners (Dell et al., 2015; Linder, 2016; van Rooij & Zirkle, 2016; Varonis, 2015), most discussions on accessibility are framed around compliance with relevant disability laws and guidelines (Lowenthal et al., 2020). However, we posit that part of the reason instructional designers, and online educators, in particular, think about accessibility in terms of compliance could be because of how popular quality assurance frameworks (e.g., Quality Matters) used by colleges and universities focus on accessibility. Instructional designers and online educators often turn to these quality assurance frameworks to help them design and develop quality online courses. However, questions remain about how well these frameworks address accessible online learning. To investigate this problem, we analyzed how popular online quality assurance frameworks and specifically the standards they are based on address accessibility for students with disabilities. In this paper, we report the results of our inquiry and conclude by discussing the implications for research and practice.

There are several laws and guidelines that online educators need to be aware of (Lowenthal et al., 2020). For instance, the Rehabilitation Act of 1973 states that programs receiving federal financial assistance, including many higher education institutions, cannot discriminate or exclude people with disabilities. Section 504 of the Rehabilitation Act specifically prohibits the denial of participation or benefit from aids or services; providing aid, benefits, or services that are not equal to others available to persons without disabilities; providing aids, benefits, or services that are not as effective; providing separate aid, benefits, or services for persons with disabilities. On the other hand, Section 508 of the Rehabilitation Act emphasizes equality in electronic and information technology developed, maintained, procured, or used by the Federal government. Section 508 provides guidance regarding accessibility, making it an important law for higher education institutions to be aware of.

The Americans with Disabilities Act (ADA) of 1990 is the most comprehensive collection of disability rights, preventing discrimination in employment, state and local governments, public accommodations, telecommunications, commercial facilities, and transportation. This act reinforces the mandate that educational entities must be nondiscrimi- natory. The ADA extended the protections outlined in ection 504 to state and local governments and thus to institutions of higher education.

In January 2017, an updated version of Section 508 of the Rehabilitation Act was published in response to market trends, innovations, and new technologies. The updated version applies standards specifically to the technology used by the federal government, including hardware, software, and websites. This update also aligned requirements with the other US and international guidelines and standards, such as the Web Content Accessibility Guidelines (WCAG).

WCAG, a World Wide Web Consortium (W3C) subgroup, is a continuous collaborative effort to provide internationally shared standards for web content accessibility (W3C, n.d.). The most recent version of WCAG contained four main principles: perceivable, operable, understandable, and robust. WCAG is a technical group of standards intended for use by web developers. It is not considered an introduction to accessibility for novice users; however, the WCAG’s website provides many examples and resources (W3C, n.d.).

Educational institutions need to understand and follow the legal directives identified in Section 504 and Section 508 of the Rehabilitation Act and the ADA. To meet or exceed compliance, educational institutions must conform to the technical standards of Section 508 as well as the most recent version of WCAG. As such, institutions, specifically instructional designers and online educators, are becoming more familiar with and complying with these basic requirements to meet the needs of diverse learners.

While the needs of students with disabilities can mirror the needs of their peers, they can also differ significantly (Rose et al., 2006; Soonhwa et al., 2019). For example, most quality assurance frameworks include standards focused on the importance of online courses to have logical and consistent course navigation (Baldwin et al., 2018). However, while a disorganized course may annoy students without disabilities, it may cause an insurmountable barrier for students with disabilities (Gladhart, 2010). Students with disabilities who experience issues with cognition, memory, and/or processing (Murphy et al., 2019) and students using assistive technology (e.g., a screen reader) to navigate courses (Whitney, 2020) may be unable to effectively locate and access content and activities effectively in disorganized courses. Research suggests that such barriers to learning have led to lower participation, retention, and graduation rates for students with disabilities in higher education (Gladhart, 2010; Izzo et al., 2008). One way to address barriers like these is to focus on designing courses that are more usable, accessible, and inclusive (Lomellini & Lowenthal, in press; Lowenthal et al., 2020).

Models of disability have changed significantly in recent years. Previously, disability was understood in medical terms as an individual’s lack of ability to do something due to health issues (World Health Organization, 2011). Following the medical model, institutions of higher education traditionally placed the onus on students to disclose their disabilities or identify what was wrong with them for the institution to help find a fix for their deficits (Nieminen & Pesonen, 2020). In this view, accommodations need to be made for individual students, on a case-by-case basis, to address the barriers created by their disability.

In contrast, a newer model, called the social model of disability, shifts the focus away from students’ medical conditions by acknowledging that learning environments create barriers, too (Nieminen & Pesonen, 2020). In the social model of disability, a course’s design and materials need to be altered instead of the student (Kuman & Wideman, 2014). Our inquiry was grounded in the theoretical lens of critical disability studies and the social model of disability. For this study, accessibility was defined as strategies to remove the barriers to access, participation, and learning for students with disabilities.

Universal Design for Learning (UDL) is a frequently cited strategy to create accessible courses found in many quality assurance frameworks. UDL is centered around three principles: providing multiple means of engagement, representation, and action and expression (CAST, 2021; Rose et al., 2006). UDL encourages a proactive approach to improve the design of a course for all learners instead of taking a reactive accommodation or remediation-focused approach for individual students (Rose et al., 2006). Taking a UDL perspective also shifts the focus from students’ disabilities to the benefits of inclusive design for diverse groups (e.g., sight, mobility, hearing, cognition, language learners, technical) (Izzo et al., 2008; Rose et al., 2006).

More than ever before, instructional designers and online educators, and the programs they work for, rely on quality assurance frameworks to create high-quality online learning experiences (Conklin et al., 2020; Zimmerman et al., 2020). Several different quality assurance frameworks are available for instructional designers and online educators to use (Baldwin et al., 2018). Some of the more popular quality assurance frameworks include the following:

• Blackboard Exemplary Course Program Rubric (ECP);

• Open SUNY Course Quality Review Scorecard (OSCQR); and

• Quality Matters (QM).

Each of these three frameworks addresses accessibility in its own way. For instance, Quality Matters is one of the most widely used quality assurance frameworks in the United States at the college and university levels (Shattuck et al., 2014). QM is centered around the following eight general standards, one of which focuses on accessibility and usability:

1. course overview and introduction;

2. learning objectives;

3. assessment and measurement;

4. instructional materials;

5. learning activities and learner interaction;

6. course technology;

7. learner support; and

8. accessibility and usability.

Most online educators, especially those in traditional faculty roles, lack knowledge and experience creating accessible online courses (Linder et al., 2015; Singleton et al., 2019). Even instructional designers (i.e., those in charge of designing and supporting faculty to develop online courses) report varying knowledge, skills, and commitment to accessibility (Singleton et al., 2019). Given this, quality assurance frameworks like QM, and specifically their focus on accessibility, serve an essential role in helping institutions deliver accessible online courses. However, we have found in our collective experience that many of these frameworks do not go far enough to both educate and support online educators and instructional designers to create usable, accessible, and inclusive online courses.

We set out in this exploratory study to analyze how online quality assurance frameworks address accessibility. More specifically, we sought to answer the following research question: How do popular online quality assurance frameworks, specifically the standards they are based on, address accessibility? While the three previously mentioned frameworks are arguably the most widely used (see Baldwin et al., 2018), we wanted to take a broader approach for this study, analyzing as many frameworks that were widely used as we could. The four researchers involved in this study were able to identify the following 13 quality assurance frameworks to analyze collectively:

1. Course Quality Assessment Rubric (CQAR), American Council of Education (2015);

2. Exemplary Course Program (ECP) Rubric, Blackboard (2016);

3. Rubric for Online Instruction (ROI), California State University (2009), Chico;

4. National Standards for Quality Online Courses (NSQOC; 2019), The International Association for K-12 Online Learning;

5. Quality Course Teaching and Instructional Practice, Online Learning Consortium (QCTIP; 2016);

6. Open SUNY Course Quality Review (OSCQR) Scorecard, SUNY Online;

7. Quality Learning and Teaching (QLT-SF State) Evaluation Instrument, San Francisco State University (2017);

8. Quality Matters (QM) Higher Ed Course Design Rubric, Quality Matters;

9. Quality Online Course Initiative Rubric, Illinois Online Network (2015; QOCI);

10. Quality Online Learning and Teaching (QLT-CSU), California State University (2015);

11. Course Rubric (CR), University of Central Florida (n.d.);

12. Rubric for Evaluating Online Courses (REOC), University of North Dakota (2016); and

13. Online Course Evaluation Guidelines (OCEG), University of Wisconsin, La Crosse (n.d.).

All researchers involved in this study, experienced online educators, then independently analyzed each quality assurance framework using the following guiding questions focused on how each framework addressed accessibility:

1. Reviewer name?

2. Quality assurance framework/rubric reviewed?

3. What year was the rubric/framework written/created?

4. What is the general focus of the rubric?

5. Does the framework/rubric address accessibility?

6. What is the name of the section focused on accessibility?

7. List the standards on accessibility.

8. What are the strengths of the accessibility standards?

9. What are the weaknesses of the accessibility standards?

10. On a scale from weak to strong, rate this framework’s approach to accessibility.

11. If we were to create our own accessibility standards, is there anything you might copy or use from this framework/ rubric?

12. Additional comments?

Then using Question 10, "On a scale from weak to strong, rate this framework’s approachto accessibility," we calculated an average of all reviewers’ responses so that we could rank, compare, and discuss each framework. We compared the individual ratings to ensure there were not any extreme outliers. We then met in a focus group format to discuss the quality assurance frameworks, our individual ratings, and reach a consensus about the rankings and each framework’s strengths and weaknesses.

Table 1 lists the average score each rubric received. OSCQR was ranked the highest with a score of 8.00 out of 10 points. QM was ranked second with a score of 7.50, followed by NSQOC and QLT-SFU tied at 7.00. QLT- CSU received a score of 6.67, and Blackboard ECP Rubric scored 6.00. These top six evaluated rubrics are discussed in more detail in the following section.

Open SUNY Course Quality Review Scorecard (OSCQR)

The Open SUNY Course Quality Review (OSCQR, 2015) scorecard, dashboard, and process are a collaboration between the Online Learning Consortium (OLC) and the Open SUNY Center for Online Teaching and Excellence. The OSCQR rubric we analyzed included 50 standards grouped into six categories:

1. course overview and information;

2. course technology and tools;

3. design and layout;

4. content and activities;

5. interaction; and

6. assessment and feedback.

While the term accessibility was used once in the rubric and did not have its own category or section, accessibility-related standards were dispersed throughout the rubric. Of the six sections, four sections-namely Sections 2, 3, 4, and 6-had items that addressed different aspects of accessibility. The reviewers found 20 standards throughout the rubric related to accessibility. This rubric sparked a debate among the reviewers about whether a quality assurance (QA) rubric should have a dedicated section on accessibility. On one side, a dedicated accessibility section brings attention to the critical nature of designing inclusive online learning. Alternatively, a separate accessibility section, especially one included at the end of the rubric, may reinforce a misguided approach to addressing accessibility at the end of the design process. After it has been designed, removing content is often much more laborious and complicated than designing accessible content and activities from the start. The implications of separating accessibility in its own section or incorporating it throughout a quality rubric warrant further research.

The OSCQR scorecard (i.e., the rubric) was found to be the most comprehensive framework reviewed. The standards were specific and actionable. There were guidelines for formatting text, designing tables, using color contrast, chunking content with ample white space, creating hyperlinks, including transcripts and captions, displaying content in HTML, and more. Each standard is linked to more information, explanations, evidence, and examples in a complimentary website (https:// oscqr.suny.edu). The accompanying website referenced specific laws (i.e., Americans with Disabilities Act (ADA), Section 504 and 508 of the Rehabilitation Act of 1973, and New York State Enterprise IT Policy NYS-P08- 005). It also provided resources related to WCAG and UDL, as mentioned earlier, a framework intended to optimize learning and reduce barriers for all learners (CAST, 2021). The reviewers discussed that explicitly referencing WCAG and UDL within the rubric could provide course designers with a deeper understanding of the foundation, intent, and implementation strategies for the standards.

Quality Matters (QM) was created in 2003 with a Fund for the Improvement of Postsecondary Education (FIPSE) grant from the U.S. Department of Education (Zimmerman et al., 2020). QM is now its own international not- for-profit organization that focuses on creating and revising standards, processes, and professional development related to quality in online and blended learning (Quality Matters, 2018b). Unlike OSCQR’s open model, QM requires a paid subscription to access their detailed QM Rubric Workbook that contains annotations and examples.

We reviewed the sixth edition of the Quality Matters Higher Education Rubric (2018a), which contained eight general standards and forty-two specific standards. Standard 8: Accessibility and Usability focused on the importance of utilizing UDL guidelines for quality online courses to be accessible and usable for all learners. Standard 8 consisted of the following six specific standards:

• 8.1: Course navigation facilitates ease of use. (3 points)

• 8.2: The course design facilitates readability. (3 points)

• 8.3: The course provides accessible text and images in files, documents, LMS pages, and web pages to meet the needs of diverse learners. (3 points)

• 8.4: The course provides alternative means of access to multimedia content in formats that meet the needs of diverse learners. (2 points)

• 8.5: Course multimedia facilitates ease of use. (2 points)

• 8.6: Vendor accessibility statements are provided for all technologies required in the course. (2 points)

Each standard had a corresponding point value. Three points indicated that the standard is essential, 2 points indicated a very important standard, and 1 point was an important standard. The accessibility-related specific standards represented 15% of the total possible 100 points in the rubric. For a course to meet QM review expectations, the review must have met all 3-point essential standards and received a total score of 85 or higher. In other words, a course could not meet expectations without addressing at least Standard 8.1, 8.2, and 8.3.

The reviewers found the standards and supporting material (i.e., the QM Rubric Workbook) related to accessibility were comprehensive. They covered text, documents, files, multimedia, web content, and third-party tools inside and outside the learning management system. Further, their use of terms such as usability and diverse learners may reduce the stigma often attached to accessibility. The annotated rubric workbook referred specifically to UDL guidelines as well as WCAG. There was no mention of specific U.S. federal laws, likely because QM is an international organization. The rubric clearly stated that "Meeting QM Specific Review Standards regarding accessibility does not guarantee or imply that the specific accessibility regulations of any country are met." The reviewers agreed that this type of statement should be included in all frameworks to reinforce that standards cannot replace internal testing or meeting specific students’ needs.

This version of the QM Rubric also required vendor accessibility statements for all technologies. The reviewers discussed the importance of encouraging third-party companies to create more accessible products. Institutions should also vet campus-wide technology and be wary of vendor accessibility statements that are sometimes created by marketing personnel instead of accessibility experts.

The QM Rubric available to the public is an abridged version lacking essential details. For example, specific terms in the publicly available rubric, such as ease of use, were ambiguous and would not be helpful to less experienced online educators or instructional designers. Overall, though, we noted that the rubric and the workbook lacked guidance on the technical details needed to make online courses usable and accessible. However, since our review, Quality Matters developed an accessibility resource site to address this need. QM’s publicly available Accessibility and Usability Resource Site contains videos, tutorials, and discussions moderated by accessibility experts on a wide range of topics, including UDL, alternative (alt) text, video captions, color contrast, accessible documents, and accessible design in learning management systems (Quality Matters, 2021). QM also offers professional development courses for practitioners to learn to use the rubric.

The National Standards for Quality Online Courses (NSQOC) were created by the International Association for K-12 Online Learning. We reviewed Version 2, released in October 2011, revised from an earlier 2007 rubric published by the North American Council for Online Learning. It is important to note that as of today, the NSQOC is now called the National Standards for Quality Online Learning and is maintained and updated by The Virtual Learning Leadership Alliance and QM (see https://www.nsqol.org/about/). The NSQOC rubric we reviewed included 52 standards grouped into five main sections:

1. Section A: Content;

2. Section B: Instructional Design;

3. Section C: Student Assessment;

4. Section D: Technology; and

5. Section E: Course Evaluation and Support.

Section D’s accessibility subsection had one standard that required all course materials and activities to be designed with UDL guidelines in mind and to conform to Section 504, Section 508, and WCAG 2.0. Section E’s Instructor and Student Support subsection contained a vague reference to accessibility that may refer to the availability of support for the instructor instead of supporting students with disabilities.

The reviewers found that the NSQOC rubric was vague in addressing accessibility in blended and online courses. While the Section 504, Section 508, and WCAG 2.0 requirements and guidelines were all specifically named, no examples or descriptions of those requirements and guidelines were provided. The reviewer considerations section mentioned the need to use a web-accessibility evaluation tool to assess whether all student- accessed web pages-such as the institution’s website, course materials, and grade retrieval-conformed to accessibility standards; however, the name of the tool was not included. The NSQOC rubric also highlighted using the National Instructional Materials Accessibility Standard to ensure textbooks and course materials were accessible.

NSQOC was the only rubric that suggested reviewers utilize accessibility evaluation tools to assess the level of accessibility for all webbased content within a course; OSCQR, though, did mention accessibility evaluation tools in their accompanying website but not specifically in the rubric. This rubric could have included a link or links to specific web accessibility evaluation tools to ensure all course evaluations used the same assessment tool to increase consistency and reliability. Another way to improve it might be to separate the UDL guidelines and break down the Section 504, Section 508, and WCAG 2.0 guidelines and requirements into additional standards with more detailed reviewer consideration explanations. The rubric we reviewed (which we recognized and discussed was dated at the time of our review) included no details for what those guidelines and requirements include.

San Francisco State University released Version 2 of the Quality Learning and Teaching (QLT-SF State) Evaluation Instrument in September 2017. San Francisco State University began working with California State University in 2013 to establish the QLT initiative. The purpose of the QLT initiative was to involve faculty in self and peer evaluations to measure course effectiveness. As a result of the partnership, the original version of the QLT Evaluation Instrument was released in 2013. Version 1 of the instrument contained 10 sections comprising 58 objectives, while Version 2 of the instrument contained the same 10 sections with 57 objectives. The QLT-SF State rubric we reviewed grouped 57 standards into 10 main sections.

1. course overview and introduction;

2. assessment of student learning;

3. instructional materials and resources;

4. students interaction and community;

5. facilitation and instruction (course delivery);

6. technology for teaching and learning;

7. learner support and resources;

8. accessibility and universal design;

9. course summary and wrap-up; and 10. mobile platform readiness.

Three of the six objectives in Section 8 were a part of the Core 24 required standards. The Core 24 objectives were considered the most important and were required to be met for the course to achieve a passing score. With this rubric, a Likert scale was used to rate each objective from 1 (met often or always), 0 (not met or rarely met), or NA if it did not apply. The reviewers noted several similarities between the QLT-SF State Rubric and the Quality Matters Rubric; for instance, a course must meet 85% of the objectives-including all the Core 24-to be fully certified.

The objectives in the accessibility and usability Section addressed a variety of accessibility concerns. Objectives 8.1, 8.3, and 8.4 primarily focused on ensuring that students are aware of university disability services. Objective 8.2 focused on the instructor providing diverse instructional support for a variety of learning styles and abilities. Objectives 8.5 and 8.6 focused on the use of accessible course materials and technologies in courses. The rubric included detailed examples and links to outside resources on creating accessible documents and web content. For example, one resource is linked to an Accessible Media Quick Converter to convert document files (e.g., PDFs) into accessible formats. It also included contact information for the academic technology department for additional support, including identifying and vetting third-party tools.

The reviewers found that the QLT-SF rubric was more specific in addressing accessibility in blended and online courses than many of the other rubrics reviewed. For instance, objective 8.5 described the difference between captions, audio descriptions, and transcripts, along with specific scenarios when they were and were not required. While multimedia was addressed more specifically in the QLT-SF rubric, web accessibility was not explicitly described or explained in any of the six objectives in Section 8; federal laws such as Section 504, Section 508, and technology guidelines such as WCAG 2.0 were not specifically mentioned either. As a result, accessibility concerns such as tables, color contrast, descriptive hyperlinks, and alt text for images were not addressed. Further, given the lack of evidence on learning styles (see Kirschner, 2017; Willingham et al., 2015), the reviewers thought any mention of learning styles should be removed or described as learning preferences.

California State University (CSU) developed the Quality Learning and Teaching (QLT) program (formally known as the Quality Online Learning & Teaching program). Since 2003, CSU has focused on assisting various audiences (i.e., faculty, faculty developers, and instructional designers) to effectively design and deliver online, blended, and flipped courses. The QLT-CSU Evaluation Instrument we reviewed contained 53 objectives within 10 sections.

1. Course Overview and Introduction;

2. Assessment and Evaluation of Student Learning;

3. Instructional Materials and Resources Utilized;

4. Students Interaction and Community;

5. Facilitation and Instruction;

6. Technology for Teaching and Learning;

7. Learner Support and Resources;

8. Accessibility and Universal Design;

9. Course Summary and Wrap-Up; and 10. Mobile Design Readiness.

Section 8, Accessibility and Universal Design, included seven objectives regarding accessibility. And Section 7, the Learner Support and Resources, included one objective related to supporting students with disabilities. These objectives discussed linking to university policies, demonstrating proactive support, communicating the role of the instructor, ensuring course materials/tools are accessible and providing accommodations when needed. The main strength of this rubric was the emphasis on working with disability support offices, which was not included in many of the other rubrics.

While the rubric provided an example column, the reviewers felt that it did not include enough specificity. For example, people new to accessibility may not understand what "assistive technology ready" means. When specifics were mentioned, there was not enough context or links to additional resources and guidance. The number of accessibility checks mentioned in standard 8.5 (e.g., text formatting, navigation, images, graphics, colors, audio, video) could be overwhelming without additional support. The rubric also did not mention any technical accessibility guidelines (e.g., WCAG). Objective 8.3 emphasized the importance for instructors to include a campus ADA compliance statement, but it did not specify which technical standard the course designer should strive to meet.

Overall, the QLT-CSU Evaluation Instrument provided a philosophical framework that encompasses different perspectives. However,it did not have enough details to support online educators or instructional designers who might not have the knowledge, skills, or prior experience to create accessible courses and instructional materials.

In 2000, Blackboard created the Exemplary Course Program (ECP) to identify and disseminate best practices for designing high-quality courses. The ECP includes the framework of course design, interaction and collaboration, assessment, and learner support. The ECP Rubric was designed to be used for selfreview, peer review, or expert review. The Learner Support section of the rubric we reviewed addressed accessibility within two subcategories: Technical Accessibility Issues and Accommodations for Disabilities. Technical considerations included using standardized formats and providing alternatives if students do not have access to a specific technology. One strength of the ECP Rubric was that it explicitly mentioned optimizing technology and multimedia elements, acknowledging the different internet speeds that students may use. For example, the ECP Rubric encouraged streamed videos, smaller files to download, and graphics designed for web delivery. The Accommodations for Disabilities subcategory acknowledged that courses need to include links to institutional policies and contacts and procedures for supporting learners with disabilities. That subcategory also mentioned support structures needed to ensure learners with disabilities could fully participate, provide alternative formats (e.g., transcripts), and consider design features (e.g., color, audio/ video controls, alt text). Additionally, there were guidelines of UDL in other sections throughout the rubric as well. For example, the rubric included criteria such as chunking content, providing multiple means of assessments, providing support materials, and enabling feedback mechanisms.

However, the ECP Rubric was limited in providing accessibility guidance. No specific federal laws or technical standards were referenced (i.e., Section 508, WCAG 2.0, or 2.1). A benefit of identifying technical standards is that it could direct online educators and instructional designers to more resources about the nuances of making courses accessible. More importantly, the ECP Rubric was extremely vague regarding accessibility criteria. The Technical Accessibility subcategory had good information, but it was not about students with disabilities. The Accommodations for Disabilities section did not mention specifics about ensuring that all content is searchable or what "color, text size manipulations, audio and video controls, and alt tags reflect universal design considerations" looked like in practice. There was also no mention of the need for courses to be accessible by people using adaptive technology.

Overall, the ECP Rubric addressed some basics, but it provided few specifics about how a designer would implement accessibility in a course, especially for online educators and instructional designers new to accessibility; however, it still was rated higher than seven other rubrics reviewed.

We found that the top evaluated quality assurance frameworks provided detailed explanations, annotations, links to additional resources, and references to industry laws and standards. For example, the OSCQR scorecard not only provided specific accessibility-related criteria in the framework, but it also included links to external resources to educate users on how to review and achieve each standard. Likewise, the subscriber version of the QM rubric workbook provided detailed annotations with examples of what to look for reviewing online courses (or when using the rubric as guidance during the course development process) (Quality Matters, 2018b). These additional resources can help address online educators’ need for web-based resources and support the design of inclusive courses (Gladhart, 2010; Izzo et al., 2008; Singleton et al., 2019). The actionable and specific vocabulary in the highly-rated rubrics’ standards can provide the practical support course designers (whether online educators or instructional designers) need.

All of the top-rated rubrics used Universal Design for Learning as part of their foundation to highlight the benefits of designing instruction for all learners. The overview statement for QM’s Standard 8 explicitly stated that a quality course should utilize UDL and demonstrate a commitment to ensuring all learners can access, use, navigate, and interact with the course components. Interestingly, some rubrics, such as QLT-SF State and QLT-CSU, utilized "Universal Design" in the title of the standard. In contrast, others, such as OSCQR, refer to the guidelines without naming UDL specifically in the scorecard. ECP briefly mentioned that quality courses should reflect universal accessibility considerations" without stating UDL as the conceptual framework. Specifically, naming UDL in the quality assurance frameworks could provide users with additional resources and information. The UDL Guidelines provided by CAST are easy to understand by varying levels of professionals in the field (CAST, 2021).

Similarly, the evaluators debated the effectiveness of explicitly stating applicable laws (e.g., Section 504, Section 508, ADA; except in international usage) and technical guidelines (i.e., WCAG). The inclusion of federal laws may reinforce the importance of designing inclusive learning experiences; however, faculty more often indicate a preference for training in UDL as opposed to training related to federal mandates (Izzo et al., 2008). Likewise, references to WCAG in quality assurance frameworks could provide online educators and instructional designers with more accessibility-related technical guidance; simultaneously, WCAG’s complicated technical standards could be overwhelming and confusing for novices.

Overall, all the top six quality assurance frameworks addressed accessibility directly. OSCQR, the highest-rated rubric, incorporated accessibility throughout the rubric, while most other rubrics had dedicated accessibility standards and/or sections. We discussed the strengths and weaknesses of both approaches. On one side, including accessibility throughout the rubric emphasized the importance of integrating accessibility into each phase of course design and stresses the interconnectedness of accessibility with other areas of quality course design. The integrated approach may also make accessibility more digestible-and thus more effective-by breaking down the information into smaller, more manageable steps (Lomellini & Lowenthal, in press; Singleton et al., 2019). Some research has suggested that avoiding terms like accessibility and UDL in favor of inclusive design, for example, can be a more successful approach to obtain faculty buy-in (Singleton et al., 2019, pp. 223-224). However, since faculty often cite UDL as the desired topic for training (Izzo et al., 2008), having a specific UDL-related section in a quality assurance framework may bring a necessary awareness to course designers who are less familiar with accessibility. This debate warrants further research.

Alternatively, we found that the lower-rated rubrics were missing key information about why accessibility matters and how to design inclusive blended and online learning environments effectively. Course designers with limited accessibility knowledge may be unaware of the potential for course design creating barriers for students with disabilities (Fitchen et al., 2009). Including information about the importance of accessibility could motivate course designers to make applicable changes (Dallas et al., 2014; Izzo et al., 2008). However, to achieve effective change that supports diverse learners, inclusive design strategies must be easy to implement (Lomellini & Lowenthal, in press). For a quality assurance framework to be adopted and used effectively, it needs to include specific examples and resources to support practical applications.

The lower rated rubrics often had vague language and jargon that would limit the usability for reviewers (e.g., faculty). For instance, research has shown that faculty can become frustrated and overwhelmed by trying to meet the needs of diverse, technologically savvy learners (Izzo et al., 2008). The sheer amount of accessibility-related information and technical skills required can be overwhelming and lead to inaction (Linder et al., 2015). Including explanations for why accessibility is important and how to meet technical standards may make the frameworks more relatable, usable, and understandable for course designers.

We set out to analyze how online quality assurance frameworks address accessibility. Our analysis illustrated how different popular quality assurance frameworks focused on accessibility. QA frameworks and corresponding rubrics rated highly included detailed information that made practical application of the accessibility standards possible. Many of the higher-rated rubrics included links to external resources and explicitly mentioned technical guidelines (e.g., WCAG) and conceptual frameworks (UDL). However, those rated poorly used vague language and often contained an overwhelming number of topics with little support or guidance for practical application.

One of the many lessons we learned reviewing these was how important it is to be intentional about the language used. While definitions are debated and evolving, accessibility typically relates to removing barriers for people with disabilities, while UDL represents a broader approach of proactively reducing barriers for all learners (Persson et al., 2015). Specific and actionable verbiage in each standard would make the frameworks more usable for a wide range of users, from novice to expert.

We hope that our results will help organizations improve their rubrics directly and/ordevelop supplemental resources as QM has since our initial analysis. The highlighted strengths and weaknesses could be used to improve the existing quality rubrics. Based on our findings, providing resources and details about why and how to meet the stated criteria would increase the usability of the frameworks. When creating or testing any framework, it is essential to include people with disabilities throughout all phases of the project. Likewise, practitioners with varying levels of accessibility knowledge should review the framework to ensure it is understandable to all users.

While accessibility in online higher education is a popular topic, we contend that it can also be more complex than a quality assurance framework can address with a series of standards. For instance, accessibility standards can quickly become too technical for the average user. Quality assurance frameworks are intended to be succinct reminders of essential information to consider when designing online or blended courses (Lowenthal & Davidson- Shivers, 2019). They are not intended to encompass all training and resources needed to implement particular standards or create accessible courses. Reducing the complex concept of inclusive course design to a few brief standards can emphasize a compliance mentality when more holistic training is needed. The increasing reliance on using rubrics to create usable, accessible, and inclusive courses across U.S. higher education institutions reinforces the idea that accessibility is a set of boxes to check off some checklist. Attempts at compliance to a minimum set of guidelines may not be enough to remove barriers created by an inaccessible course design; however, ultimately, more research is needed.