Skip to main content
SpringerLink
Log in

InPlaceKV: in-place update scheme for SSD-based KV storage systems under update-intensive Worklaods

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

Modern key-value (KV) storage systems adopt append-only writes to update KV pairs with the out-of-place manner, because the performance of sequential accesses is much better than that of random accesses for HDDs. Compaction or GC operations will be deployed by traditional KVs or KV separation schemes due to updating KV pairs via append-only writes. Unfortunately, the system performance will be hurt because extra reads and writes are triggered during those operations, especially under update-intensive workloads.We find that the performance gap for SSDs between sequential and random accesses will get close when the request size becomes large in our experiments. Motivated by this, we propose InPlaceKV built atop SSDs, which adopts an in-place large-update scheme with a hotness-aware method to update KV pairs rather than use append-only writes with the LSM-tree. We further design the working flow of system operations with appropriate data structures. Finally, we compare InPlaceKV with state-of-the-art KV storage systems via extensive experiments under update-intensive workloads, and results validate the effectiveness of our design in improving the system throughput.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data availability

The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

References

  1. Chemawat S, Dean J.:Leveldb. https://github.com/google/leveldb, (2022)

  2. Facebook. Rocksdb. http://rocksdb.org/, (2022)

  3. Redis, Sanfilippo S.: https://redis.io, (2022)

  4. Huang, Dongxu, Liu, Qi., Cui, Qiu, Fang, Zhuhe, Ma, Xiaoyu, Fei, Xu., Shen, Li., Tang, Liu, Zhou, Yuxing, Huang, Menglong, Wei, Wan, Liu, Cong, Zhang, Jian, Li, Jianjun, Xuelian, Wu., Song, Lingyu, Sun, Ruoxi, Shuaipeng, Yu., Zhao, Lei, Cameron, Nicholas, Pei, Liquan, Tang, Xin: Tidb: a raft-based htap database. Proc. VLDB Endow 13(12), 3072–3084 (2020)

    Article  Google Scholar 

  5. Elyasi, N., Choi, C., Sivasubramaniam, A.: Large-scale graph processing on emerging storage devices. In Proceedings of the 17th USENIX Conference on File and Storage Technologies, pages 309–316. USENIX Association, (2019)

  6. Bronson, N., Amsden, Z., Cabrera, G., Chakka, P., Venkataramani, V.: Tao: Facebook’s distributed data store for the social graph. In Proceedings of the 2013 USENIX Conference on Annual Technical Conference, pages 49–60. USENIX Association, (2013)

  7. Chang, Fay, Dean, Jeffrey, Ghemawat, Sanjay, Hsieh, Wilson C., Wallach, Deborah A., Burrows, Mike, Chandra, Tushar, Fikes, Andrew, Gruber, Robert E.: Bigtable: a distributed storage system for structured data. Acm Trans. Comput. Syst. 26(2), 1–26 (2008)

    Article  Google Scholar 

  8. Raju, Pandian., Kadekodi, Rohan., Chidambaram, Vijay., Abraham, Ittai.: Pebblesdb: Building key-value stores using fragmented log-structured merge trees. In Proceedings of the 26th Symposium on Operating Systems Principles, pages 497–514. Association for Computing Machinery, (2017)

  9. Dayan, Niv., Idreos, Stratos.: Dostoevsky: Better space-time trade-offs for lsm-tree based key-value stores via adaptive removal of superfluous merging. In Proceedings of the 2018 International Conference on Management of Data, pages 505–520. Association for Computing Machinery, (2018)

  10. Lu, Lanyue., Pillai, Thanumalayan Sankaranarayana., Arpaci-Dusseau, Andrea C., Arpaci-Dusseau, Remzi H.: Wisckey: Separating keys from values in ssd-conscious storage. In Proceedings of the 14th USENIX Conference on File and Storage Technologies, pages 133–148. USENIX Association, (2016)

  11. li, Yongkun., Liu, Zhen., Lee, Patrick P.C., Wu, Jiayu., Xu, Yinlong., Wu, Yi., Tang, Liu., Liu, Qi., Cui, Qiu.: Differentiated key-value storage management for balanced i/o performance. In Proceedings of the 2021 USENIX Annual Technical Conference, pages 673–687. USENIX Association, (2021)

  12. Chan, Helen H. W., Li, Yongkun., Lee, Patrick P. C., Xu, Yinlong.: Hashkv: Enabling efficient updates in KV storage via hashing. In Proceedings of the 2018 USENIX Annual Technical Conference, pages 1007–1019. USENIX Association, (2018)

  13. Tang, Chenlei, Wan, Jiguang, Changsheng, Xie: Fencekv: enabling efficient range query for key-value separation. IEEE Trans. Parallel Distrib Syst. 33(12), 3375–3386 (2022)

    Article  Google Scholar 

  14. O’Neil, P., Cheng, E., Gawlick, D., O’Neil, E.: The log-structured merge-tree (lsm-tree). Acta Informatica 33(4), 351–385 (1996)

    Article  Google Scholar 

  15. TPC. Tpc-c is an on-line transaction processing. http://www.tpc.org/tpcc/, (2022)

  16. Kavalanekar, S., Worthington, B., Zhang, Q., Sharda, V.: Characterization of storage workload traces from production windows servers. In 2008 IEEE International Symposium on Workload Characterization, pp. 119–128. IEEE, Piscataway (2008)

    Book  Google Scholar 

  17. Lepers, Baptiste., Balmau, Oana., Gupta, Karan., Zwaenepoel, Willy.: Kvell: The design and implementation of a fast persistent key-value store. page 447-461, (2019)

  18. Yu, Geoffrey X., Markakis, Markos, Kipf, Andreas, Larson, Per-Åke, Minhas, Umar Farooq, Kraska, Tim: Treeline: an update-in-place key-value store for modern storage. Proc. VLDB Endow. 16(1), 99–112 (2022)

    Article  Google Scholar 

  19. Zhang, Jianpeng, Lin, Mingwei, Pan, Yubiao, Xu, Zeshui: Crftl: cache reallocation-based page-level flash translation layer for smartphones, pp. 1–9. Piscataway, IEEE Transactions on Consumer Electronics (2023)

    Google Scholar 

  20. Luo, Yuhan, Lin, Mingwei, Pan, Yubiao, Zeshui, Xu.: Dual locality-based flash translation layer for nand flash-based consumer electronics. IEEE Trans. Consumer Electron. 68(3), 281–290 (2022)

    Article  Google Scholar 

  21. Pan, Yubiao, Lin, Mingwei, Zhixiong, Wu., Zhang, Huizhen, Zeshui, Xu.: Caching-aware garbage collection to improve performance and lifetime for nand flash ssds. IEEE Trans. Consumer Electron. 67(2), 141–148 (2021)

    Article  Google Scholar 

  22. Pan, Yubiao, Li, Yongkun, Zhang, Huizhen, Chen, Hao, Lin, Mingwei: Gftl: Group-level mapping in flash translation layer to provide efficient address translation for nand flash-based ssds. IEEE Trans. Consumer Electron. 66(3), 242–250 (2020)

    Article  Google Scholar 

  23. Pugh, William W.: Skip lists: a probabilistic alternative to balanced trees. Commun. ACM 33(6), 668–676 (1990)

    Article  Google Scholar 

  24. Bayer, R., Mccreight, E.M.: Organization and maintenance of large ordered indexes. Acta Informatica 1(3), 173–189 (1972)

    Article  Google Scholar 

  25. Cooper, B. F., Silberstein, A., Tam, E., Ramakrishnan, R., Sears, R.: Benchmarking cloud serving systems with ycsb. In Proceedings of the 1st ACM Symposium on Cloud Computing. Indianapolis, pages 143–154. Association for Computing Machinery, (2010)

  26. Ren, Kai, Zheng, Qing, Arulraj, Joy, Gibson, Garth: Slimdb: a space-efficient key-value storage engine for semi-sorted data. Proc. VLDB Endow. 10(13), 2037–2048 (2017)

    Article  Google Scholar 

  27. Pan, F., Yue, Y., Xiong, J.: dcompaction: delayed compaction for the lsm-tree. Int. J. Parallel Program. 45(6), 1310–1325 (2017)

    Article  Google Scholar 

  28. Shetty, P., Spillane, R., Malpani, R., Andrews, B., Zadok, E.: Building workload-independent storage with vt-trees. In Proceedings of the 11th USENIX conference on File and Storage Technologies, pages 17–30. USENIX Association, (2013)

  29. Mei, F., Cao, Q., Jiang, H., Li, J.: Sifrdb: A unified solution for write-optimized key-value stores in large datacenter. In Proceedings of the 2018 ACM Symposium on Cloud Computing, page 477-489. Association for Computing Machinery, (2018)

  30. Sears, Russell., Ramakrishnan, Raghu.: Blsm: A general purpose log structured merge tree. In Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, pages 217–228. Association for Computing Machinery, (2012)

  31. Dayan, Niv., Athanassoulis, Manos., Idreos, Stratos.: Monkey: Optimal navigable key-value store. In Proceedings of the 2017 ACM International Conference on Management of Data, pages 79–94. Association for Computing Machinery, (2017)

  32. Li Yongkun, Tian, Chengjin., Guo, Fan., Li, Cheng., Xu, Yinlong.: Elasticbf: Elastic bloom filter with hotness awareness for boosting read performance in large key-value stores. In Proceedings of the 2019 USENIX Annual Technical Conference, pages 739–752. USENIX Association, (2019)

  33. Byun, Hayoung, Lim, Hyesook: Learned fbf: Learning-based functional bloom filter for key-value storage. IEEE Trans. Comput. 71(8), 1928–1938 (2022)

    Google Scholar 

  34. Wu, Fenggang., Yang, Ming-Hong., Zhang, Baoquan., Du, David H. C.: Ac-key: Adaptive caching for lsm-based key-value stores. In Proceedings of the 2020 USENIX Annual Technical Conference, pages 603–615. USENIX Association, (2020)

  35. Kai, Lu., Zhao, Nannan, Wan, Jiguang, Fei, Changhong, Zhao, Wei, Deng, Tongliang: Tridentkv: a read-optimized lsm-tree based kv store via adaptive indexing and space-efficient partitioning. IEEE Trans. Parallel Distrib. Syst. 33(8), 1953–1966 (2022)

    Article  Google Scholar 

  36. Luo, Siqiang., Chatterjee, Subarna., Ketsetsidis, Rafael., Dayan, Niv., Qin, Wilson., Idreos, Stratos.: Rosetta: A robust space-time optimized range filter for key-value stores. In Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data, pages 2071–2086. Association for Computing Machinery, (2020)

  37. Zhong, W., Chen, C., Wu, X., Jiang, S.: Remix: Efficient range query for lsm-trees. In Proceedings of the 19th USENIX Conference on File and Storage Technologies, pages 51–64. USENIX Association, (2021)

  38. Ge, Xuran., Liu, Yang., Wu, Lizhou., Ou, Yang., Chen, Zhiguang., Xiao, Nong.: Pm-based persistent key value stores: a survey. pages 1–7, (2022)

  39. Kaiyrakhmet, O., Lee, S., Nam, B., Noh, S. H., Choi, Y.: Slm-db: single-level key-value store with persistent memory. In Proceedings of the 17th USENIX Conference on File and Storage Technologies, pages 191–205. USENIX Association, (2019)

  40. Chen, Youmin., Lu, Youyou., Yang, Fan., Wang, Qing., Wang, Yang., Shu, Jiwu.: Flatstore: An efficient log-structured key-value storage engine for persistent memory. In Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems, pages 1077–1091. Association for Computing Machinery, (2020)

  41. Li, Wen-Jie., Jiang, Dejun., Xiong, Jin., Bao, Yungang.: Hilsm: an lsm-based key-value store for hybrid NVM-SSD storage systems. In Proceedings of the 17th ACM International Conference on Computing Frontiers, pages 208–216. Association for Computing Machinery, (2020)

  42. Chen, Hao., Ruan, Chaoyi., Li, Cheng., Ma, Xiaosong., Xu, Yinlong.: Spandb: A fast, cost-effective lsm-tree based KV store on hybrid storage. In Proceedings of the 19th USENIX Conference on File and Storage Technologies, pages 17–32. USENIX Association, (2021)

  43. Yao, Ting., Wan, Jiguang., Huang, Ping., Zhang, Yiwen., Liu, Zhiwen., Xie, Changsheng., He, Xubin.: Geardb: A gc-free key-value store on HM-SMR drives with gear compaction. In Proceedings of the 17th USENIX Conference on File and Storage Technologies, pages 159–171. USENIX Association, (2019)

  44. Conway, Alexander., Gupta, Abhishek., Chidambaram, Vijay., Farach-Colton, Martin., Spillane, Richard P.: Amy Tai, and Rob Johnson. Splinterdb: Closing the bandwidth gap for nvme key-value stores. In Proceedings of the 2020 USENIX Annual Technical Conference, pages 49–63. USENIX Association, (2020)

  45. Ray, Madhurima., Kant, Krishna., Li, Peng., Trika, Sanjeev.: Flashkey: A high-performance flash friendly key-value store. In Proceedings of the 2020 IEEE International Parallel and Distributed Processing Symposium, pages 976–985. IEEE, (2020)

  46. Menon, Prashanth, Qadah, Thamir M., Rabl, Tilmann, Sadoghi, Mohammad, Jacobsen, Hans-Arno.: Logstore: A workload-aware, adaptable key-value store on hybrid storage systems. IEEE Trans. Knowl. Data Eng. 34(8), 3867–3882 (2022)

    Article  Google Scholar 

  47. Chen, Yen-Ting., Yang, Ming-Chang., Chang, Yuan-Hao., Shih, Wei-Kuan.: Parallel-log-single-compaction-tree: Flash-friendly two-level key-value management in kvssds. In Proceedings of the 25th Asia and South Pacific Design Automation Conference, pages 277–282. IEEE, (2020)

  48. Wu, Sung-Ming., Lin, Kai-Hsiang., Chang, Li-Pin.: KVSSD: close integration of LSM trees and flash translation layer for write-efficient KV store. In Proceedings of the 2018 Design, Automation & Test in Europe Conference & Exhibition, pages 563–568. IEEE, (2018)

Download references

Funding

This research work was supported by the National Natural Science Foundation of China under Grants No. 61802133 and No. 61872086, the Young Top Talent of Young Eagle Program of Fujian Province, China under Grant No. F21E0011202B01, the Natural Science Foundation of Fujian Province under Grants No. 2021J01319 and No. 2022J06020, and the Fundamental Research Founds for the Central Universities of Huaqiao University under Grant No. ZQN-910.

Author information

Authors and Affiliations

  1. The School of Computer Science and Technology, Huaqiao University, Xiamen, 361000, Fujian, China

    Jianing Zhao, Yubiao Pan & Huizhen Zhang

  2. The College of Mathematics and Informatics, Fujian Normal University, Fuzhou, 350000, Fujian, China

    Mingwei Lin

  3. The College of Computer and Information Science, Southwest University, Chongqing, 400000, Chongqing, China

    Xin Luo

  4. The Business School, Sichuan University, Chengdu, 610000, Sichuan, China

    Zeshui Xu

Authors
  1. Jianing Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yubiao Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Huizhen Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mingwei Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xin Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zeshui Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization: JZ, YP, HZ; Methodology: JZ, YP, HZ; Investigation: JZ, YP, HZ; Formal analysis: JZ; Software: JZ; Validation: JZ; Writing - original draft: JZ; Supervision: YP, ML; Writing - review and editing: YP, ML, XL, ZX; Resources: ML, XL, ZX.

Corresponding authors

Correspondence to Yubiao Pan or Mingwei Lin.

Ethics declarations

Conflict of interest

The authors have no relevant financial or non-financial interests to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, J., Pan, Y., Zhang, H. et al. InPlaceKV: in-place update scheme for SSD-based KV storage systems under update-intensive Worklaods. Cluster Comput 27, 1527–1540 (2024). https://doi.org/10.1007/s10586-023-04031-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-023-04031-9

Keywords

Search

Navigation