The purpose of this paper is to illustrate and evaluate the semantic process benchmarking concept.
The authors' approach includes the use of metamodels and ontologies, which make the process models syntactically and semantically comparable. Furthermore, a software prototype is presented to analyze and compare individual process models and their performance information. Thereafter, the technical, conceptual, and economic perspectives of the approach's evaluation are aligned with their respective outcomes.
The evaluation proves that this approach is generally suitable to generate novel and useful information on different process models and their performance within the same problem domain. However, the initial set‐up costs are high and will only pay off once process models are used regularly.
The proposed approach depends strongly on the availability of appropriate metrics and ontologies, as well as on the annotation of these ontologies to process models, which is a time‐consuming task. If large benchmarking clearing centers are established, the approach will be more cost‐effective. The developed SEMAT prototype, that demonstrates and proves the proposed approach's general viability, supports cost‐effective ontology engineering and annotation in the context of semantic process benchmarking initiatives.
To date, process benchmarking has primarily been a manual process. In this article, the authors suggest an approach that allows time‐consuming and costly process analysis to be partially automated, which makes the performance indicators, as well as qualitative differences between processes, apparent.
