Rui Ding
ABSTRACT
Sketch algorithms have been widely deployed for network measurement as they achieve high accuracy with restricted resource usage. They store measurement results compactly in fixed-size counters. However, as sketch counters are skewed towards low values, higher bits in most counters remain zero. Such massive unused bits impair the space efficiency valued by sketch algorithms. Unfortunately, efforts to mitigate the issue either apply to specific algorithms or compromise accuracy. In this paper, we design BitSense, a novel optimization framework that integrates with existing sketch algorithms. The key idea is to regard higher bits in sketch counters as a sparse vector and leverage compressive sensing techniques to compress and restore counters. Further, BitSense provides a programming model to help developers easily realize sketch algorithms without dealing with the details of compression and recovery. Bit-Sense proposes an automatic approach for parameter configuration. It theoretically guarantees nearly zero error under the configuration. We have built a BitSense prototype in P4 and a software platform and integrated it with fourteen sketch solutions. Extensive experiments show that BitSense significantly reduces the memory usage of existing sketch solutions by 25%-80% while incurring little overhead and almost zero accuracy drop, outperforming five state-of-the-art optimization frameworks.
- Behnaz Arzani, Selim Ciraci, Luiz Chamon, Yibo Zhu, Hongqiang Harry Liu, Jitu Padhye, Boon Thau Loo, and Geoff Outhred. 2018. 007: Democratically finding the cause of packet drops. In Proc. of NSDI. 419--435.Google Scholar
- Behnaz Arzani, Selim Ciraci, Boon Thau Loo, Assaf Schuster, and Geoff Outhred. 2016. Taking the blame game out of data centers operations with netpoirot. In Proc. of SIGCOMM. 440--453.Google Scholar
- Ran Ben Basat, Gil Einziger, Michael Mitzenmacher, and Shay Vargaftik. 2021. SALSA: self-adjusting lean streaming analytics. In Proc. of ICDE. IEEE, 864--875.Google ScholarCross Ref
- Theophilus Benson, Aditya Akella, and David A Maltz. 2010. Network traffic characteristics of data centers in the wild. In Proc. of ACM IMC. 267--280.Google ScholarDigital Library
- Burton H Bloom. 1970. Space/time trade-offs in hash coding with allowable errors. Commun. ACM 13, 7 (1970), 422--426.Google ScholarDigital Library
- Flavio Bonomi, Michael Mitzenmacher, Rina Panigrahy, Sushil Singh, and George Varghese. 2006. An improved construction for counting bloom filters. In Algorithms-ESA 2006: 14th Annual European Symposium, Zurich, Switzerland, September 11--13, 2006. Proceedings 14. Springer, 684--695.Google Scholar
- caida [n.d.]. The CAIDA UCSD Statistical information for the CAIDA Anonymized Internet Traces. https://www.caida.org/data/passive/passive_trace_statistics.Google Scholar
- Emmanuel J Candès, Justin K Romberg, and Terence Tao. 2006. Stable signal recovery from incomplete and inaccurate measurements. Communications on Pure and Applied Mathematics 59, 8 (2006), 1207--1223.Google ScholarCross Ref
- Emmanuel J Candès and Terence Tao. 2006. Near-optimal signal recovery from random projections: Universal encoding strategies? IEEE Transactions on Information Theory 52, 12 (2006), 5406--5425.Google ScholarDigital Library
- Emmanuel J Candès and Michael B Wakin. 2008. An introduction to compressive sampling. IEEE signal processing magazine 25, 2 (2008), 21--30.Google ScholarCross Ref
- Moses Charikar, Kevin Chen, and Martin Farach-Colton. 2002. Finding frequent items in data streams. In International Colloquium on Automata, Languages, and Programming. Springer, 693--703.Google ScholarDigital Library
- Min Chen, Shigang Chen, and Zhiping Cai. 2016. Counter tree: A scalable counter architecture for per-flow traffic measurement. IEEE/ACM ToN 25, 2 (2016), 1249--1262.Google ScholarDigital Library
- Yi-Chao Chen, Lili Qiu, Yin Zhang, Guangtao Xue, and Zhenxian Hu. 2014. Robust network compressive sensing. In Proc. of MobiCom. 545--556.Google ScholarDigital Library
- Chun Tung Chou, Rajib Rana, and Wen Hu. 2009. Energy efficient information collection in wireless sensor networks using adaptive compressive sensing. In Proc. of LCN. IEEE, 443--450.Google ScholarCross Ref
- Saar Cohen and Yossi Matias. 2003. Spectral bloom filters. In Proc. of SIGMOD. 241--252.Google ScholarDigital Library
- Graham Cormode and Shan Muthukrishnan. 2005. An improved data stream summary: the count-min sketch and its applications. Journal of Algorithms 55, 1 (2005), 58--75.Google ScholarDigital Library
- Graham Cormode and Shan Muthukrishnan. 2005. Summarizing and mining skewed data streams. In Proc. of SDM. SIAM, 44--55.Google Scholar
- Graham Cormode and Shanmugavelayutham Muthukrishnan. 2005. What's new: Finding significant differences in network data streams. IEEE/ACM ToN 13, 6 (2005), 1219--1232.Google ScholarDigital Library
- Olivier Dubois and Jacques Mandler. 2002. The 3-XORSAT threshold. Comptes Rendus Mathématique 335, 11 (2002), 963--966.Google ScholarCross Ref
- Gil Einziger and Roy Friedman. 2016. Counting with tinytable: Every bit counts!. In Proc. of ICDCN. 1--10.Google ScholarDigital Library
- Cristian Estan and George Varghese. 2002. New directions in traffic measurement and accounting. In Proc. of SIGCOMM. 323--336.Google ScholarDigital Library
- Rodrigo Fonseca, Tianrong Zhang, Karl Deng, and Lihua Yuan. 2019. dShark: A general, easy to program and scalable framework for analyzing in-network packet traces. (2019), 207--220.Google Scholar
- Andreas Goerdt and Lutz Falke. 2012. Satisfiability Thresholds beyond k-XORSAT. In International Computer Science Symposium in Russia. Springer, 148--159.Google ScholarCross Ref
- Junzhi Gong, Tong Yang, Yang Zhou, Dongsheng Yang, Shigang Chen, Bin Cui, and Xiaoming Li. 2017. Abc: a practicable sketch framework for non-uniform multisets. In Proc. of IEEE International Conference on Big Data. 2380--2389.Google ScholarCross Ref
- Gaël Guennebaud, Benoît Jacob, et al. 2010. Eigen v3. http://eigen.tuxfamily.org.Google Scholar
- Arpit Gupta, Rob Harrison, Marco Canini, Nick Feamster, Jennifer Rexford, and Walter Willinger. 2018. Sonata: Query-driven streaming network telemetry. In Proc. of SIGCOMM. 357--371.Google ScholarDigital Library
- Nan Hua, Bill Lin, Jun Xu, and Haiquan Zhao. 2008. Brick: A novel exact active statistics counter architecture. In Proc. of ANCS. 89--98.Google ScholarDigital Library
- Qun Huang, Xin Jin, Patrick PC Lee, Runhui Li, Lu Tang, Yi-Chao Chen, and Gong Zhang. 2017. Sketchvisor: Robust network measurement for software packet processing. In Proc. of SIGCOMM. 113--126.Google ScholarDigital Library
- Qun Huang, Patrick PC Lee, and Yungang Bao. 2018. Sketchlearn: relieving user burdens in approximate measurement with automated statistical inference. In Proc. of SIGCOMM. 576--590.Google ScholarDigital Library
- Qun Huang, Siyuan Sheng, Xiang Chen, Yungang Bao, Rui Zhang, Yanwei Xu, and Gong Zhang. 2021. Toward {Nearly-Zero-Error} Sketching via Compressive Sensing. In Proc. of NSDI. 1027--1044.Google Scholar
- Gabriel Istrate, Stefan Boettcher, and Allon G Percus. 2005. Spines of random constraint satisfaction problems: definition and connection with computational complexity. Annals of Mathematics and Artificial Intelligence 44, 4 (2005), 353--372.Google ScholarDigital Library
- Lavanya Jose, Minlan Yu, and Jennifer Rexford. 2011. Online Measurement of Large Traffic Aggregates on Commodity Switches.. In Workshop on Hot-ICE.Google Scholar
- Changhoon Kim, Anirudh Sivaraman, Naga Katta, Antonin Bas, Advait Dixit, and Lawrence J Wobker. 2015. In-band network telemetry via programmable dataplanes. In Proc. of SIGCOMM, Vol. 15.Google Scholar
- Gene Moo Lee, Huiya Liu, Young Yoon, and Yin Zhang. 2005. Improving sketch reconstruction accuracy using linear least squares method. In Proc. of ACM IMC. 24--24.Google ScholarCross Ref
- Haoyu Li, Qizhi Chen, Yixin Zhang, Tong Yang, and Bin Cui. 2022. Stingy sketch: a sketch framework for accurate and fast frequency estimation. Proc. of VLDB 15, 7 (2022), 1426--1438.Google ScholarDigital Library
- Jizhou Li, Zikun Li, Yifei Xu, Shiqi Jiang, Tong Yang, Bin Cui, Yafei Dai, and Gong Zhang. 2020. Wavingsketch: An unbiased and generic sketch for finding top-k items in data streams. In Proc. of SIGKDD. 1574--1584.Google ScholarDigital Library
- Tao Li, Shigang Chen, and Yibei Ling. 2012. Per-flow traffic measurement through randomized counter sharing. IEEE/ACM ToN 20, 5 (2012), 1622--1634.Google ScholarDigital Library
- Yuliang Li, Rui Miao, Changhoon Kim, and Minlan Yu. 2016. FlowRadar: A Better NetFlow for Data Centers. In Proc. of NSDI. 311--324.Google Scholar
- Yuliang Li, Rui Miao, Hongqiang Harry Liu, Yan Zhuang, Fei Feng, Lingbo Tang, Zheng Cao, Ming Zhang, Frank Kelly, Mohammad Alizadeh, et al. 2019. HPCC: High precision congestion control. In Proc. of SIGCOMM. 44--58.Google ScholarDigital Library
- Yang Li, Hao Wu, Tian Pan, Huichen Dai, Jianyuan Lu, and Bin Liu. 2016. Case: Cache-assisted stretchable estimator for high speed per-flow measurement. In Proc. of INFOCOM. IEEE, 1--9.Google ScholarDigital Library
- Kate Ching-Ju Lin and Wei-Lun Lai. 2022. MC-Sketch: Enabling Heterogeneous Network Monitoring Resolutions with Multi-Class Sketch. In Proc. of INFOCOM. IEEE, 220--229.Google Scholar
- Lingtong Liu, Yulong Shen, Yibo Yan, Tong Yang, Muhammad Shahzad, Bin Cui, and Gaogang Xie. 2020. Sf-sketch: a two-stage sketch for data streams. IEEE TPDS 31, 10 (2020), 2263--2276.Google Scholar
- Zaoxing Liu, Ran Ben-Basat, Gil Einziger, Yaron Kassner, Vladimir Braverman, Roy Friedman, and Vyas Sekar. 2019. Nitrosketch: Robust and general sketch-based monitoring in software switches. In Proc. of SIGCOMM. 334--350.Google ScholarDigital Library
- Zaoxing Liu, Antonis Manousis, Gregory Vorsanger, Vyas Sekar, and Vladimir Braverman. 2016. One sketch to rule them all: Rethinking network flow monitoring with UnivMon. In Proc. of SIGCOMM. 101--114.Google ScholarDigital Library
- Zaoxing Liu, Hun Namkung, Georgios Nikolaidis, Jeongkeun Lee, Changhoon Kim, Xin Jin, Vladimir Braverman, Minlan Yu, and Vyas Sekar. 2021. Jaqen: A {High-Performance} {Switch-Native} Approach for Detecting and Mitigating Volumetric {DDoS} Attacks with Programmable Switches. In 30th USENIX Security Symposium (USENIX Security 21). 3829--3846.Google Scholar
- Yi Lu, Andrea Montanari, Balaji Prabhakar, Sarang Dharmapurikar, and Abdul Kabbani. 2008. Counter braids: a novel counter architecture for per-flow measurement. ACM SIGMETRICS Performance Evaluation Review 36, 1 (2008), 121--132.Google ScholarDigital Library
- Yi Lu and Balaji Prabhakar. 2009. Robust counting via counter braids: An error-resilient network measurement architecture. In Proc. of INFOCOM. IEEE, 522--530.Google ScholarCross Ref
- Ahmed Metwally, Divyakant Agrawal, and Amr El Abbadi. 2005. Efficient computation of frequent and top-k elements in data streams. In International conference on database theory. Springer, 398--412.Google Scholar
- Rui Miao, Hongyi Zeng, Changhoon Kim, Jeongkeun Lee, and Minlan Yu. 2017. Silkroad: Making stateful layer-4 load balancing fast and cheap using switching asics. In Proc. of SIGCOMM. 15--28.Google ScholarDigital Library
- Hun Namkung, Zaoxing Liu, Daehyeok Kim, Vyas Sekar, and Peter Steenkiste. 2023. Sketchovsky: Enabling Ensembles of Sketches on Programmable Switches. In Proc. of NSDI. 1273--1292.Google Scholar
- p4 2022. P4 Language. http://p4.org.Google Scholar
- Boris Pittel and Gregory B Sorkin. 2016. The satisfiability threshold for k-XORSAT. Combinatorics, Probability and Computing 25, 2 (2016), 236--268.Google ScholarCross Ref
- Jiuhua Qi, Wenjun Li, Tong Yang, Dagang Li, and Hui Li. 2019. Cuckoo counter: A novel framework for accurate per-flow frequency estimation in network measurement. In Proc. of ANCS. IEEE, 1--7.Google ScholarCross Ref
- Sriram Ramabhadran and George Varghese. 2003. Efficient implementation of a statistics counter architecture. In Proc. of the ACM SIGMETRICS international conference on Measurement and modeling of computer systems. 261--271.Google ScholarDigital Library
- Pratanu Roy, Arijit Khan, and Gustavo Alonso. 2016. Augmented sketch: Faster and more accurate stream processing. In Proc. of SIGMOD. 1449--1463.Google ScholarDigital Library
- Devavrat Shah, Sundar Iyer, Balaji Prabhakar, and Nick McKeown. 2001. Analysis of a statistics counter architecture. In HOT 9 Interconnects. Symposium on High Performance Interconnects. IEEE, 107--111.Google ScholarCross Ref
- Siyuan Sheng, Qun Huang, Sa Wang, and Yungang Bao. 2021. PR-sketch: monitoring per-key aggregation of streaming data with nearly full accuracy. Proc. of VLDB 14, 10 (2021), 1783--1796.Google ScholarDigital Library
- Anirudh Sivaraman, Alvin Cheung, Mihai Budiu, Changhoon Kim, Mohammad Alizadeh, Hari Balakrishnan, George Varghese, Nick McKeown, and Steve Licking. 2016. Packet transactions: High-level programming for line-rate switches. In Proc. of SIGCOMM. 15--28.Google ScholarDigital Library
- Vibhaalakshmi Sivaraman, Srinivas Narayana, Ori Rottenstreich, Shan Muthukrishnan, and Jennifer Rexford. 2017. Heavy-hitter detection entirely in the data plane. In Proc. of ACM SOSR. 164--176.Google ScholarDigital Library
- Cha Hwan Song, Pravein Govindan Kannan, Bryan Kian Hsiang Low, and Mun Choon Chan. 2020. FCM-sketch: generic network measurements with data plane support. In Proc. of CoNEXT. 78--92.Google ScholarDigital Library
- Lu Tang, Qun Huang, and Patrick PC Lee. 2019. Mv-sketch: A fast and compact invertible sketch for heavy flow detection in network data streams. In Proc. of INFOCOM. IEEE, 2026--2034.Google ScholarDigital Library
- tofino 2022. Barefoot's Tofino. https://www.barefootnetworks.com/technology.Google Scholar
- Yibo Yan, Cheng Chen, Huiping Lin, Olivier Ruas, Tengjiao Wang, and Tong Yang. 2020. Priority-aware per-flow measurement using cuckoo sketch. In Proc. of IFIP Networking Conference (Networking). IEEE, 622--624.Google Scholar
- Mingran Yang, Alex Baban, Valery Kugel, Jeff Libby, Scott Mackie, Swamy Sadashivaiah Renu Kananda, Chang-Hong Wu, and Manya Ghobadi. 2022. Using trio: juniper networks' programmable chipset-for emerging in-network applications. In Proc. of SIGCOMM. 633--648.Google ScholarDigital Library
- Tong Yang, Siang Gao, Zhouyi Sun, Yufei Wang, Yulong Shen, and Xiaoming Li. 2019. Diamond sketch: Accurate per-flow measurement for big streaming data. IEEE TPDS 30, 12 (2019), 2650--2662.Google Scholar
- Tong Yang, Jie Jiang, Peng Liu, Qun Huang, Junzhi Gong, Yang Zhou, Rui Miao, Xiaoming Li, and Steve Uhlig. 2018. Elastic sketch: Adaptive and fast network-wide measurements. In Proc. of SIGCOMM. 561--575.Google ScholarDigital Library
- Tong Yang, Haowei Zhang, Jinyang Li, Junzhi Gong, Steve Uhlig, Shigang Chen, and Xiaoming Li. 2019. HeavyKeeper: An Accurate Algorithm for Finding Top-k Elephant Flows. IEEE/ACM ToN 27, 5 (2019), 1845--1858.Google ScholarDigital Library
- Tong Yang, Yang Zhou, Hao Jin, Shigang Chen, and Xiaoming Li. 2017. Pyramid sketch: A sketch framework for frequency estimation of data streams. Proc. of VLDB 10, 11 (2017), 1442--1453.Google ScholarDigital Library
- Minlan Yu. 2019. Network telemetry: towards a top-down approach. ACM SIGCOMM Computer Communication Review 49, 1 (2019), 11--17.Google ScholarDigital Library
- Minlan Yu, Lavanya Jose, and Rui Miao. 2013. Software {Defined}{Traffic} Measurement with {OpenSketch}. In Proc. of NSDI. 29--42.Google Scholar
- Ying Zhang. 2013. An adaptive flow counting method for anomaly detection in SDN. In Proc. of CoNEXT. 25--30.Google ScholarDigital Library
- Bohan Zhao, Xiang Li, Boyu Tian, Zhiyu Mei, and Wenfei Wu. 2021. DHS: Adaptive Memory Layout Organization of Sketch Slots for Fast and Accurate Data Stream Processing. In Proc. of SIGKDD. 2285--2293.Google ScholarDigital Library
- Qi Zhao, Jun Xu, and Zhen Liu. 2006. Design of a novel statistics counter architecture with optimal space and time efficiency. In Proc. of the joint international conference on Measurement and modeling of computer systems. 323--334.Google ScholarDigital Library
- Xiaolei Zhao, Mei Wen, Minjin Tang, Qun Huang, and Chunyuan Zhang. 2020. HybridSketch: A Memory-centric Precise Approach for Flow Measurement. In Proc. of ICC. IEEE, 1--7.Google ScholarCross Ref
- Yikai Zhao, Kaicheng Yang, Zirui Liu, Tong Yang, Li Chen, Shiyi Liu, Naiqian Zheng, Ruixin Wang, Hanbo Wu, Yi Wang, et al. 2021. {LightGuardian}: A {full-visibility}, lightweight, in-band telemetry system using sketchlets. In 18th USENIX Symposium on Networked Systems Design and Implementation (NSDI 21). 991--1010.Google Scholar
- Yang Zhou, Hao Jin, Peng Liu, Haowei Zhang, Tong Yang, and Xiaoming Li. 2018. Accurate per-flow measurement with bloom sketch. In Proc. of NFOCOM WKSHPS. IEEE, 1--2.Google ScholarCross Ref
- Yang Zhou, Peng Liu, Hao Jin, Tong Yang, Shoujiang Dang, and Xiaoming Li. 2017. One memory access sketch: a more accurate and faster sketch for per-flow measurement. In Proc. of GLOBECOM. IEEE, 1--6.Google ScholarDigital Library
- Yang Zhou, Ying Zhang, Minlan Yu, Guangyu Wang, Dexter Cao, Eric Sung, and Starsky Wong. 2022. Evolvable Network Telemetry at Facebook. In Proc. of NSDI. USENIX Association.Google Scholar
- Yibo Zhu, Nanxi Kang, Jiaxin Cao, Albert Greenberg, Guohan Lu, Ratul Mahajan, Dave Maltz, Lihua Yuan, Ming Zhang, Ben Y Zhao, et al. 2015. Packet-level telemetry in large datacenter networks. In Proc. of SIGCOMM. 479--491.Google ScholarDigital Library
Index Terms
- BitSense: Universal and Nearly Zero-Error Optimization for Sketch Counters with Compressive Sensing
Recommendations
Image compressive sensing via Truncated Schatten-p Norm regularization
Low-rank property as a useful image prior has attracted much attention in image processing communities. Recently, a nonlocal low-rank regularization (NLR) approach toward exploiting low-rank property has shown the state-of-the-art performance in ...
Compressive sensing via nonlocal low-rank tensor regularization
The aim of Compressing sensing (CS) is to acquire an original signal, when it is sampled at a lower rate than Nyquist rate previously. In the framework of CS, the original signal is often assumed to be sparse and correlated in some domain. Recently, ...
Block compressive sensing
Iterative re-weighted l1 norm minimization reconstruction algorithm.Joint reconstruction algorithm for multiview images and video frames.The correlation between multiple images is exploited through joint sparsity model. Compressive sensing provides ...
Comments