Gartner claims the ever increasing demand for real-time analytics requires the fusion of Transactional (OLTP) and Analytical (OLAP) databases, eschewing ETL processes and giving birth to the so-called Hybrid Analytical and Trans- actional Processing (HTAP) databases.
Unfortunately, current benchmarking approaches are not able to comprehensively produce a unified metric from the assessment of a HTAP database. The evaluation of both database systems is done separately, leading to the use of disjoint sets of benchmarks such as TPC-C or TPC-H.
In this paper we propose a new benchmark, HTAPBench, providing a unified metric for HTAP databases geared towards the execution of constantly increasing OLAP requests limited by an admissible impact on OLTP performance. To achieve this, a load balancer within HTAPBench regulates the coexistence of OLTP and OLAP workloads, proposing a method for the generation of both new data and requests, so that OLAP requests over freshly modified data are comparable across runs.
We demonstrate the merit of our approach by using different types of databases OLTP, OLAP and HTAP; showing that the benchmark is able to highlight the differences be- tween them, while producing queries with comparable complexity across runs and negligible variability on the size of the result sets.