– This paper aims to propose a framework for the evaluation of artefacts in Design Science and test it using an exemplar case of a knowledge management system (KMS) developed for an Australian refrigeration manufacturing company.
– The research uses Design Science research methodology in a specific case study context. The artefact studied was developed using an ontology based on an engineering design conceptualisation and created using an ontology generator, Protégé. Research data for the evaluation of the framework were collected using a combination of document analysis, interviews, shadowing and observations.
– The evaluation framework developed for the research and applied to the KMS specifically built for the company was shown to be useful in determining the efficacy and effectiveness of the research outcomes in terms of usefulness to the company engineers in the technical analysis of their work, and for the CEO and COO as part of their strategic planning for the company. The evaluation framework helped the researcher and the engineers as collaborators to demonstrate the extent of improvement in the design and build processes in the company.
– Prior research in both Information System and Design Science has not provided a specific, generalizable, evaluation framework for system developers to use as a guide during the systems development process. This research proposes an evaluation framework which covers all broad aspects of evaluation and efficacy, accepting that evaluation frameworks must be flexible in enabling changes to accommodate variations in the types and purposes of artefacts developed.
