Posts Tagged ‘TPC-C
XML Database Benchmarks
Semi-structured data models and query languages have been widely studied. Several storage strategies and mapping schemes for XML data using a relational database are explored. Domain-specific database benchmarks for OLTP (TPC-C), decision support (TPC-H, TPC-R, APB-1), information retrieval, spatial data management (Sequoia) etc. are available. XOO7, XMach-1 and XMark are the three benchmarks currently available that test XMLMS for their query processing abilities.
Table 1. Comparing Benchmarks over XML system characteristics
XOO7 design attempts to harness the similarities in data models of XML and object-oriented approaches. Although XML attempts to provide a framework for handling semistructured data, it encompasses most of the modeling features ofcomplex object models. There are straight forward correspondences between the object-oriented schemas and instances and XML DTDs and data. XOO7 is an adaptation of the OO7 Benchmark for object-oriented database systems. XOO7 provides 18 query challenges. The current implementation of XOO7 tests XML management systems which store their data locally.
XMach-1 tests multi-user features provided by the systems. The benchmark is modeled for a web application using XML data. It evaluates standard and non-standard linguistic features such as insertion, deletion, querying URL and aggregate operations. Although the proposed workload and queries are interesting, the benchmark has not been applied and no results have been published yet. XMark developed under the XML benchmark project at CWI, is a benchmark proposed for XML data stores. The benchmark consists of an application scenario which models an Internet auction site and 20 XQuery challenges designed to cover the essentials of XML query processing. These queries have been evaluated on an internal research prototype, Monet XML, to give a first baseline.
Table 1 compares the expressive power of queries from XOO7, XMark and XMach-1. As can be seen XOO7 is the most comprehensive benchmark in terms of XML functionalities covered. Both XMark and XMach-1 focus on a datacentric usage of XML. All three benchmarks provide queries to test relational model characteristics like selection, projection and reduction. Properties like transaction processing, view manipulation, aggregation and update, are not yet tested by any of the benchmarks. XMach-1 covers delete and insert operations, although the semantics of such operations are yet to be clearly defined under XML query model.
XOO7 is a comprehensive benchmark as can be seen from Table 1 and also empirical evaluations show the ability of the XOO7 queries to distinguish all the desired functionalities supported by an XML database. In the absence of queries exploiting the document-centric features, XMark and XMach-1 may not be able to clearly distinguish XML-enabled systems from Native XML management systems.