Skip to main content

Advertisement

SpringerLink
Log in

Measuring performances and footprint of blockchains with BCTMark: a case study on Ethereum smart contracts energy consumption

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

A rich ecosystem of blockchain-based projects has emerged since the introduction of Bitcoin in 2008. New protocols seek to improve the performances of blockchain systems. In particular, the energy consumption of blockchains has been particularly decried. Unfortunately, those new proposals are often evaluated with ad hoc tools and experimental environments. Therefore, reproducibility and comparison of these new contributions with the state of the art of blockchain technologies are complicated. To the best of our knowledge, only a few tools partially address the design of a generic benchmarking of blockchain technologies (e.g., load generation). This paper introduces BCTMark, a generic framework for benchmarking blockchain technologies on an emulated network in a reproducible way. Based on this novel framework, we studied a key aspect of modern blockchains’ energy consumption: smart-contract execution. Based on experiments and the analysis of one year of real-world Ethereum transactions, we measured and modeled smart-contracts’ energy consumption on Ethereum. Furthermore, this study details how the replication of contract calls execution can impact their energy cost. In particular, we give insights on the energy consumed by smart-contracts on Ethereum over one year.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Notes

  1. see BIPs 100 to 107 on the evolution of Bitcoin block size and emission rates

  2. A batch of transactions

  3. The presence of bytecode is the main difference between a contract account and a “human” account

  4. Bootnodes are peers that have an address known by everyone in the network. New peers can connect to those bootnodes to get the address of other peers in the network

  5. By workload, we mean transactions to be processed by the system under test

  6. CPU usage values seem to differ from ones in Fig. 5 but this visual effect is due to 1) high variance in data and 2) high density in data points that hides lowest values. We can notice the high variance on Fig. 5.

  7. Ethereum Virtual Machine, the VM running smart-contracts

  8. Calculated on https://ethgasstation.info with average gas price

  9. Each server is equiped with a sensor that measure its energy consumption

  10. In fact, the second version of Ethereum, in development, will include a Proof-of-Stake engine that does not include any mining.

  11. Infrastructure and deployment protocol is the same than the one described in Sect. 5.2. Smart-contract deployed is Quicksort.

  12. As detailed in Sect. 5.3, around 140 million contract calls were found in the one-year dataset we extracted

  13. SUT system under test

References

  1. Steem: Steem—an incentivized, blockchain-based, public content platform. https://steem.com/steem-whitepaper.pdf. whitepaper (2016)

  2. E-Estonia. https://e-estonia.com/

  3. Labs, P.: Filecoin: A Decentralized Storage Network. https://filecoin.io/filecoin.pdf. whitepaper (2017)

  4. Storj Labs, I.: Storj: a decentralized cloud storage network framework. https://storj.io/storjv3.pdf. whitepaper (2018)

  5. Mylrea, M., Gourisetti, S.N.G.: Blockchain for smart grid resilience: Exchanging distributed energy at speed, scale and security. In: 2017 Resilience Week (RWS), pp. 18–23 (2017). IEEE

  6. Mengelkamp, E., Gärttner, J., Rock, K., Kessler, S., Orsini, L., Weinhardt, C.: Designing microgrid energy markets: a case study: the brooklyn microgrid. Appl. Energy 210, 870–880 (2018)

    Article  Google Scholar 

  7. Eyal, I., Gencer, A.E., Sirer, E.G., Van Renesse, R.: Bitcoin-ng: a scalable blockchain protocol. In: 13th \(\{USENIX\}\) symposium on networked systems design and implementation (\(\{NSDI\}\) 16), pp. 45–59 (2016)

  8. Decker, C., Wattenhofer, R.: A fast and scalable payment network with bitcoin duplex micropayment channels. In: Symposium on self-stabilizing systems, pp. 3–18 (2015). Springer

  9. Vranken, H.: Sustainability of bitcoin and blockchains. Curr. Opin. Environ. Sustain. 28, 1–9 (2017)

    Article  Google Scholar 

  10. O’Dwyer, K.J., Malone, D.: Bitcoin Mining and Its Energy Footprint. IEEE (2014)

  11. Saleh, F.: Blockchain without waste: Proof-of-stake. Available at SSRN 3183935 (2019)

  12. Rocket, T.: Snowflake to avalanche: a novel metastable consensus protocol family for cryptocurrencies. IPFS (2018)

  13. Poon, J., Dryja, T.: The bitcoin lightning network: scalable off-chain instant payments (2016)

  14. Balouek, D., Carpen Amarie, A., Charrier, G., Desprez, F., Jeannot, E., Jeanvoine, E., Lèbre, A., Margery, D., Niclausse, N., Nussbaum, L., Richard, O., Pérez, C., Quesnel, F., Rohr, C., Sarzyniec, L.: Adding virtualization capabilities to the Grid’5000 testbed. In: Ivanov, I.I., van Sinderen, M., Leymann, F., Shan, T. (eds.) Cloud Computing and Services Science. Communications in Computer and Information Science, vol. 367, pp. 3–20. Springer, Berlin (2013). https://doi.org/10.1007/978-3-319-04519-1_1

  15. Smaalders, B.: Performance anti-patterns. ACM Queue 4(1), 44–50 (2006)

    Article  Google Scholar 

  16. Grafana—the open observable platform. https://grafana.com/. Accessed 6 Dec, 2019

  17. Saingre, D., Ledoux, T., Menaud, J.-M.: Bctmark: a framework for benchmarking blockchain technologies. In: 2020 IEEE/ACS 17th international conference on computer systems and applications (AICCSA), pp. 1–8 (2020). IEEE

  18. PoET 1.0 Specification. https://sawtooth.hyperledger.org/docs

  19. Foundation, E.: A Next-Generation Smart Contract and Decentralized Application Platform. https://github.com/ethereum/wiki/wiki/White-Paper. whitepaper (2013)

  20. Saraph, V., Herlihy, M.: An empirical study of speculative concurrency in ethereum smart contracts. arXiv preprint arXiv:1901.01376 (2019)

  21. Gavin, W.: Ethereum: a secure decentralised generalised transaction ledger. https://ethereum.github.io/yellowpaper/paper.pdf. yellowpaper (2020)

  22. Ethereum Wiki—Ethash. https://github.com/ethereum/wiki/wiki/Ethash

  23. Lal, C., Marijan, D.: Blockchain testing: vhallenges, techniques, and research directions. arXiv preprint arXiv:2103.10074 (2021)

  24. Zheng, P., Zheng, Z., Luo, X., Chen, X., Liu, X.: A detailed and real-time performance monitoring framework for blockchain systems. In: 2018 IEEE/ACM 40th international conference on software engineering: software engineering in practice track (ICSE-SEIP), pp. 134–143 (2018). IEEE

  25. Alharby, M., van Moorsel, A.: Blocksim: a simulation framework for blockchain systems. ACM SIGMETRICS Performance Eval. Rev. 46(3), 135–138 (2019)

    Article  Google Scholar 

  26. Zander, M., Waite, T., Harz, D.: Dagsim: simulation of dag-based distributed ledger protocolss. ACM SIGMETRICS Performance Eval. Rev. 46(3), 118–121 (2019)

    Article  Google Scholar 

  27. Permenev, A., Dimitrov, D., Tsankov, P., Drachsler-Cohen, D., Vechev, M.: Verx: Safety verification of smart contracts. In: 2020 IEEE symposium on security and privacy (SP), pp. 1661–1677 (2020). IEEE

  28. Enoslib: Enoslib: a framework to build experimental frameworks on various platforms. https://github.com/BeyondTheClouds/enoslib. Github repository

  29. Ansible: Ansible is Simple IT Automation. https://www.ansible.com/

  30. Telegraf—the plugin-driven server agent for collecting and reporting metrics. https://github.com/influxdata/influxdb. Accessed 6 Dec, 2019

  31. InfluxDB—scalable datastore for metrics, events, and real-time analytics. https://github.com/influxdata/telegraf. Accessed 6 Dec, 2019

  32. Locust—a modern load testing framework. https://locust.io/. Accessed 6 Dec, 2019

  33. Pastor, J., Menaud, J.M.: Seduce: a testbed for research on thermal and power management in datacenters. In: 2018 26th international conference on software, telecommunications and computer networks (SoftCOM), pp. 1–6 (2018). IEEE

  34. Dinh, T.T.A., Wang, J., Chen, G., Liu, R., Ooi, B.C., Tan, K.-L.: Blockbench: a framework for analyzing private blockchains. In: Proceedings of the 2017 ACM international conference on management of data, pp. 1085–1100 (2017). ACM

  35. McCook, H.: An order-of-magnitude estimate of the relative sustainability of the bitcoin network. A critical assessment of the Bitcoin mining industry, gold production industry, the legacy banking system, and the production of physical currency 2, 25 (2014)

  36. Kim, S.K., Ma, Z., Murali, S., Mason, J., Miller, A., Bailey, M.: Measuring ethereum network peers. In: Proceedings of the internet measurement conference 2018, pp. 91–104 (2018)

  37. Cooper, B.F., Silberstein, A., Tam, E., Ramakrishnan, R., Sears, R.: Benchmarking cloud serving systems with ycsb. In: Proceedings of the 1st ACM symposium on cloud computing, pp. 143–154 (2010). ACM

  38. Cahill, M.J., Röhm, U., Fekete, A.D.: Serializable isolation for snapshot databases. ACM Trans. Database Syst. (TODS) 34(4), 20 (2009)

    Article  Google Scholar 

  39. Go-Ethereum—Official Go implementation of the Ethereum protocol. https://github.com/ethereum/go-ethereum. Accessed 6 Dec, 2019

  40. Parity—fast and feature-rich multi-network Ethereum client. https://github.com/paritytech/parity-ethereum. Accessed 6 Dec, 2019

  41. Hyperledger Fabric. https://github.com/hyperledger/fabric. Accessed 6 Dec, 2019

  42. Hyperledger Caliper. https://www.hyperledger.org/projects/caliper. Accessed 6 Dec, 2019

  43. Prometheus—from metrics to insight. https://prometheus.io/. Accessed 6 Dec, 2019

  44. Pan, H., Duan, X., Wu, Y., Tseng, L., Aloqaily, M., Boukerche, A.: Bbb: A lightweight approach to evaluate private blockchains in clouds. In: GLOBECOM 2020-2020 IEEE global communications conference, pp. 1–6 (2020). IEEE

  45. Mininet: An instant Virtual Network on your laptop (or other PC). http://mininet.org/. Accessed 15 June, 2021

  46. Dong, Z., Zheng, E., Choon, Y., Zomaya, A.Y.: Dagbench: a performance evaluation framework for dag distributed ledgers. In: 2019 IEEE 12th international conference on cloud computing (CLOUD), pp. 264–271 (2019). IEEE

  47. Popov, S.: The tangle

  48. LeMahieu, C.: Nano: a feeless distributed cryptocurrency network. Nano [Online resource]. URL: https://nano.org/en/whitepaper. Accessed 24 March, 2018

  49. Churyumov, A.: Byteball: a decentralized system for storage and transfer of value. https://byteball.org/Byteball.pdf (2016)

  50. Pinna, A., Ibba, S., Baralla, G., Tonelli, R., Marchesi, M.: A massive analysis of ethereum smart contracts empirical study and code metrics. IEEE Access 7, 78194–78213 (2019)

    Article  Google Scholar 

  51. Chen, T., Li, X., Luo, X., Zhang, X.: Under-optimized smart contracts devour your money. In: 2017 IEEE 24th international conference on software analysis, evolution and reengineering (SANER), pp. 442–446 (2017). IEEE

  52. Feist, J., Grieco, G., Groce, A.: Slither: a static analysis framework for smart contracts. In: 2019 IEEE/ACM 2nd international workshop on emerging trends in software engineering for blockchain (WETSEB), pp. 8–15 (2019). IEEE

  53. Di Angelo, M., Salzer, G.: A survey of tools for analyzing ethereum smart contracts. In: 2019 IEEE international conference on decentralized applications and infrastructures (DAPPCON), pp. 69–78 (2019). IEEE

Download references

Acknowledgements

Experiments presented in this paper were carried out using the Grid’5000 testbed, supported by a scientific interest group hosted by Inria and including CNRS, RENATER and several Universities as well as other organizations (see https://www.grid5000.fr).

This paper has been financed by the HYDDA FSN project.

Funding

This paper has been financed by the HYDDA FSN project.

Author information

Authors and Affiliations

  1. Inria, LS2N, IMT Atlantique, 4 Rue Alfred Kastler, 44307, Nantes, France

    Dimitri Saingre, Thomas Ledoux & Jean-Marc Menaud

Authors
  1. Dimitri Saingre
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thomas Ledoux
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jean-Marc Menaud
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Dimitri Saingre participated in the design of the contribution, design of experiments, implementation of the contribution, implementation and run of experiments, writing of the paper, proofreading of the paper

Thomas Ledoux and Jean-Marc Menaud participated in the design of the contribution, design of experiments, writing of the paper, proofreading of the paper

Corresponding author

Correspondence to Dimitri Saingre.

Ethics declarations

Conflict of interest

Not applicable

Consent for publications

All authors consent for this paper to be published in Cluster Computing Journal

Code availability

BCTMark’s code is open-source and accessible here: https://gitlab.inria.fr/dsaingre/bctmark.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Saingre, D., Ledoux, T. & Menaud, JM. Measuring performances and footprint of blockchains with BCTMark: a case study on Ethereum smart contracts energy consumption. Cluster Comput 25, 2819–2837 (2022). https://doi.org/10.1007/s10586-021-03441-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-021-03441-x

Keywords

Search

Navigation