A study on industry and synthetic standard benchmarks in relational and object databases

Benchmarks are the vital tools in the performance measurement and evaluation of database management systems (DBMS), including the relational database management systems (RDBMS) and the object‐oriented/object‐relational database management systems (OODBMS/ORDBMS). Standard synthetic benchmarks have been used to assess the performance of RDBMS software. Other benchmarks have been utilized to appraise the performance of OODBMS/ORDBMS products. In this paper, an analytical framework of workload characterization to extensively and expansively examine the rationale and design of the industry standard and synthetic standard benchmarks is presented. This analytical framework of workload analysis is made up of four main components: the schema analysis, the operation analysis, the control analysis, and the system analysis. These analysis results are compiled and new concepts and perspectives of benchmark design are collated. Each analysis aspect is described and each managerial implication is discussed in detail.