Skip to main content
SpringerLink
Log in

Benchmarking on intensive transaction processing

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

Benchmarks play a crucial role in database performance evaluation, and have been effectively promoting the development of database management systems. With critical transaction processing requirements of new applications, we see an explosion of innovative database technologies for dealing with highly intensive transaction workloads (OLTP) with the obvious characteristics of sharp dynamics, terrificskewness, high contention, or high concurrency (abbr. DSC2), which can not be well described or evaluated by current standard benchmarks. In this paper, based on the representative SecKill applications, we define a pacakge of workloads simulating intensive transactional processing requirements. And we create a general and flexible benchmark framework PeakBench for evaluating intensive OLTP workloads on databases. We are the first work to have full control on simulating DSC2, especially for the fine granularity control for contention generation. With a comprehensive set of experiments conducted on popular open sourced DBMSs compared with the other representative OLTP benchmarks, we completely demonstrate the usefulness of PeakBench.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Difallah D E, Pavlo A, Curino C, CudreMaurouxet P. OLTP-bench: an extensible testbed for benchmarking relational databases. Proceedings of the VLDB Endowment, 2013, 7(4): 277–288

    Article  Google Scholar 

  2. Ren K, Faleiro J M, Abadi D J. Design principles for scaling multi-core OLTP under high contention. In: Proceedings of the 2016 International Conference on Management of Data. 2016, 1583–1598

  3. Zhou N, Zhou X, Zhang X, Du X, Wang S. Reordering transaction execution to boost high-frequency trading applications. Data Science and Engineering, 2017, 2(4): 301–315

    Article  Google Scholar 

  4. Persico V, Pescapé A, Picariello A, Sperli G. Benchmarking big data architectures for social networks data processing using public cloud platforms. Future Generation Computer Systems, 2018, 89: 98–109

    Article  Google Scholar 

  5. Harding R, Van Aken D, Pavlo A, Stonebraker M. An evaluation of distributed concurrency control. Proceedings of the VLDB Endowment, 2017, 10(5): 553–564

    Article  Google Scholar 

  6. MySQL. For traditional databases, details are published on the MySql home page

  7. PostgreSQL. Database details are published on the PostgreSQL home page

  8. VoltDB, LLC. Voltdb technical overview. Whitepaper, 2010

  9. Bitton D, Dewitt D J, Turbyfill C. Benchmarking database systems a systematic approach. In: Proceedings of the 9th International Conference on Very Large Data Bases. 1983, 8–19

  10. Gray J. The benchmark handbook fordatabase and transaction systems. In: Proceedings of Sigmod Conference on Gray Hardavellas. 1993

  11. Tay Y C. Data generation for application-specific benchmarking. Proceedings of the VLDB Endowment, 2011, 4(12): 1–4

    Article  Google Scholar 

  12. Chen S, Ailamaki A, Athanassoulis M, Gibbons P B, Johnson R, Pandis I, Stoica R. TPC-E vs. TPC-C: characterizing the new TPC-E benchmark via an I/O comparison study. ACM Sigmod Record, 2011, 39(3): 5–10

    Article  Google Scholar 

  13. Cole R, Funke F, Giakoumakis L, Guy W, Kemper A, Krompass S, Kuno H, Nambiar R, Neumann T, Poess M, Sattler K, Seibold M, Simon E, Waas F. The mixed workload CH-benCHmark. In: Proceedings of the 4th International Workshop on Testing Database Systems. 2011

  14. Council, Transaction Processing Performance. TPC-H benchmark specification. Tcp.org Website, 2008

  15. Cahill M J, Röhm U, Fekete A D. Serializable isolation for snapshot databases. ACM Transactions on Database Systems, 2009, 34(4): 20

    Article  Google Scholar 

  16. Wolski A. TATP Benchmark Description (Version 1.0). 2009

  17. Stonebraker M. A measure of transaction processing power. In: Hellerstein J M, Stonebraker M, Readings in Database Systems, 2nd ed. Morgan Kaufmann Publishers Inc., 1994, 442–454

  18. Cooper B F, Silberstein A, Tam E, Ramakrishnan R, Sears R. Benchmarking cloud serving systems with YCSB. In: Proceedings of the 1st ACM Symposium on Cloud Computing. 2010, 143

  19. Patil S, Polte M, Kai R, Tantisiriroj W, Lin X, López J, Gibson G, Fuchs A, Rinaldi B. YCSB++: benchmarking and performance debugging advanced features in scalable table stores. In: Proceedings of the 2nd ACM Symposium on Cloud Computing. 2011

  20. Gonzalez J E, Xin R S, Dave A, Crankshaw D, Stoica I. Graphx: graph processing in a distributed dataflow framework. In: Proceedings of the 11th Usenix Conference on Operating Systems Design & Implementation. 2014

Download references

Acknowledgements

We are partially supported by the Key Program of National Natural Science Foundation of China (2018YFB1003402) and the National Natural Science Foundation of China (Grant No. 61432006).

Author information

Authors and Affiliations

  1. School of Data Science and Engineering, East China Normal University, Shanghai, 200062, China

    Chunxi Zhang, Yuming Li, Rong Zhang, Weining Qian & Aoying Zhou

Authors
  1. Chunxi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuming Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Weining Qian
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Aoying Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rong Zhang.

Additional information

Chunxi Zhang is a doctoral student of East China Normal University, under the supervision of Prof. Rong Zhang. She received her bachelor degree in software engineering from Shandong University of Science and Technology, China in 2011. Her current research interest is performance evaluation of databases, including intensive workload generation, contention simulation and isolation validation.

Yuming Li is a doctoral student of East China Normal University, under the supervision of Prof. Aoying Zhou and Rong Zhang. He received his bachelor degree in computer science from Northeastern University, China in 2014. His research interests include application-oriented database performance testing, automated database testing and distributed data management.

Rong Zhang is a member of China Computer Federation. She received her PhD degree in computer science from Fudan University, China in 2007. She joined East China Normal University, China since 2011 and is currently a professor in the university. From 2007 to 2010, she worked as an expert researcher in NICT, Japan. Her current research interests include knowledge management, distributed data management and database benchmarking.

Weining Qian is a professor and dean of the School of Data Science and Engineering, East China Normal University, China. He received his MS and PhD in computer science from Fudan University, China in 2001 and 2004, respectively. He is now serving as a standing committee member of Database Technology Committee of China Computer Federation, and committee member of ACM SIGMOD China Chapter. His research interests include scalable transaction processing, benchmarking big data systems, and management and analysis of massive datasets.

Aoying Zhou is vice President of East China Normal University (ECNU), professor of School of Data Science and Engineering, and doctoral supervisor. Before joining ECNU, China in 2008, Aoying worked for Fudan University at the Computer Science Department for 15 years. He is the winner of the National Science Fund for Distinguished Young Scholars supported by NSFC and the professorship appointment under Changjiang Scholars Program of Ministry of Education. He is now acting as a vice-director of ACM SIGMOD China and Database Technology Committee of China Computer Federation. He is serving as a member of the editorial boards VLDB Journal, WWW Journal, etc. His research interests include data management, in-memory cluster computing, big data benchmarking and performance optimization.

Electronic Supplementary Material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, C., Li, Y., Zhang, R. et al. Benchmarking on intensive transaction processing. Front. Comput. Sci. 14, 145204 (2020). https://doi.org/10.1007/s11704-019-8438-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11704-019-8438-0

Keywords

Search

Navigation