The purpose of this study is to share quality process experience from a US comprehensive university to use both direct (participation rate and assessment quality) and indirect assessment measures (assessment survey) to evaluate the quality process.
A mixed method design was used to evaluate the quality process. The qualitative design used thematic analysis to find the common themes from the assessment survey. The quantitative design included reliability and internal consistency for the survey and inter-rater reliability for the peer review.
Findings indicated that participation rate and assessment quality improved over the years. Faculty provided positive feedback about assessment website, the peer review process and feedback but negative perspectives on the assessment management software and heavy workload on assessment activities. The researcher shared the actions made based on three-year assessment results, especially how leaders have used it to align with planning and budgeting to close the institutional effectiveness loop.
This research had two limitations. The inter-rater reliability of the peer review was not high and the number of peer review reports in two years were so different, so the results could not be checked for statistical significance.
The meta-assessment results could have important implications for other universities to improve the quality process.
There have been limited studies on using both direct and indirect assessment measures to evaluate the quality process. This study uses multiple measures to provide accurate results for administrators’ decision-making in resource allocation.
