Online Assessment and Measurement: Case Studies from Higher Education, K-12 and Corporate, by Scott L. Howell, and Mary Hricko (Eds.) (Hershey, PA: Information Science Publishing 2006, 280 pages, $84.95).
Online assessment is gaining more and more attention as Web-based instruction and training become increasingly popular. Compared with assessment in the traditional face-to-face setting, online assessment has some special issues and needs that present new challenges and opportunities to educators, instructional designers and trainers. These issues include accessibility, computer interface usability, automated grading, programmed feedback, adaptive learning, student progress report, access control, and security. This volume discusses these issues through a collection of case studies in the contexts of higher education, K-12 education, and corporate settings.
The book consists of 18 chapters organized into three sections. Section 1 has eight chapters covering online assessment in higher education including online quiz tools as part of a learning management system (LMS) or standalone applications, authentic assessment, and performance-based assessment. Section 2 has five chapters covering online assessment in K-12 education with a focus on adaptive testing to promote formative assessment. Section 3 has five chapters describing distribution models of online assessment, best practices in administering online assessment in certification programs for health care professionals and military personnel, and collaborations between corporate sector and higher education.
A prominent feature of the book is the attention given to the importance of making online assessment an integral part of the learning process, particularly in chapters 7, 8, 12, 13, and 16. Chapter 7 introduces two systems that utilize Web-based technologies for out-come-based assessment. The first system col-lects data from faculty on a course-by-course basis in order to establish baseline data for a particular program. The second system tracks the performances of students and course effec-tiveness through faculty and student surveys across the 12 campuses of the Penn State sys-tem. Chapter 8 presents a computer-aided learning tool “CALM” that facilitates and assesses student learning in mathematics, with a focus on diagnostic and directed feedback to learners for formative assessment.
Chapter 12 is a report on how a CAT tool, the Washington Assessment of Student Learning (WASL), was designed in alignment to an assessment criteria framework, the Essential Academic Learning Requirements (EALR). The system generated reports with item analysis, skill gap analysis, and student proficiencies. Chapter 13 presents a study on using formative assessment to improve student performance on standardized tests. The model is called the Predictive Assessment Series (PAS), developed by ThinkLink. It uses simulations of summative assessments that can be used for classroom formative assessments.
Another prominent feature of the book is the discussion on computer-assisted adaptive learning and testing, particularly in chapters 9, 10, 11, and 16. Chapter 9 presents a study on using adaptive tests for larger number of users. A comparability study was conducted to determine whether scores from computer-aided tests would be comparable to those from paper and pencil tests. Lessons learned include information overexposure in adaptive tests, accessibility and usability, test security, and the need for an adequate item bank. In contrast with the preceding chapter, Chapter 10 has the target student population from a small school district, with a mapping system called NWEA which is a reference framework to help interpret test results and map the outcomes to learning objectives. The study recommends a long-term commitment required to verify the validity and reliability of CAT, and for the results from CAT to be integrated with and supportive of instruction.
Chapter 11 reports on a project that used adaptive math test for third-grade students with special needs, delivering quizzes across multiple classrooms and schools simultaneously. Computer-adaptive testing is believed to be helpful for students with deficiencies in reading and writing skills in traditional paper and pencil. Chapter 16 describes how the U.S. Army Distance Center used a Web-based training product to administer online tests to service people. The chapter focuses on three areas: adaptive learning, immediate feedback, and assessment security. Adaptive learning enables the center to customize learning and training for learners from different backgrounds progressing at difference paces and at different levels. The use of database allows the system to track student learning and provide consistent and timely feedback, thus providing a constructive learning environment.
A third feature of the book is its description of the current and emerging technologies for delivering online assessment. Chapter 1 describes the assessment features offered by WebCT with details on release controls, automated grading, quiz delivery options, and report functions. Chapter 3 is a report from Capella University that introduces standards and methods for measuring the quantity and quality of instructor-student interactions in the use of online discussion board. Chapter 4 tells how Hong Kong Institution of Education utilized technology to enhance authentic assessment. The technologies utilized include the creation of a virtual community, video, and discussion forum.
Chapter 5 describes the integrated use of three tools: Rubistar (a tool for creating rubrics), FileMaker (a database for tracking student information), and Microsoft Word to facilitate assessment. Chapter 6 presents a case focused on online assessment of foreign language proficiency in Arabic and Russian. The online quiz tool used is Perception for Web by QuestionMark. Practical issues were discussed and summarized for consideration.
A final feature of the book is its discussion on the management of online assessment, including the transformation from traditional paper and pencil based testing to computer-assisted testing, distribution models for online testing, and collaboration between educational institutions and corporate sectors. Chapter 2 examines issues in using WebCT Vista in place of an in-house online assessment tool, particularly system scalability and test security. The chapter also discusses how the search for and adoption of a technology solution could be influenced by political interests and administrative bureaucracies. Chapter 14 describes three distribution models for online assessment: (a) controlled distribution, (b) global distribution, and (c) ubiquitous distribution. Chapter 15 presents a case about the use of a state dental board exam through a testing service called Thomason Prometrics that operates a network of over 3,000 testing centers in 134 countries.
Chapter 17 introduces the seven C’s adopted by the Cisco Networking Academy. The seven C’s are: claims, collaboration, complexity, contextualization, computation, communication, and coordination. The seven C’s are basically about identification of learning objectives, collaboration between technical staff and instructors, task authenticity, computer automation to provide constructive feedback, reporting functions, and alignment of assessment to school curricula. Chapter 18 dis-cusses issues and variables to consider in tran-sitioning assessment from paper and pencil test to computer-based test. The chapter presents a framework for feasibility study that consists of five phases: analysis of technical and financial viability, data gathering, cost analysis, imple-mentation plan, integration and interpretation of data, and generation of feasibility report.
Overall, the book provides a good collection of case studies that depict the efforts to explore potentials of online assessment, and varied practices in the context of higher education, K-12, and corporate sections. As pointed in the Foreword by the editors, online assessment is an emerging practice new to most educators and trainers. The findings from the studies in the book are mostly premature and the practices are experimental. With various authors writing on the same topic of online assessment, the book has some content overlapping and redundancy across chapters, particularly in describing features of online assessment, such as quiz question types, automated grading, report functions, and security issues. Nevertheless, the case studies in the book can serve as useful resources that readers can refer to in planning, designing, implementing, and evaluating online assessment.
