Abstract
Artificial intelligence based methods for operations of IT-systems (AIOps) support the process of maintaining and operating large IT infrastructures on different levels, e.g. anomaly detection, root cause analysis, or initiation of self-stabilizing activities. The foundation for the deployment of such methods are extensive and reliable metric data on the current state of the overall system. In particular, network information expressing the core parameters latency, throughput, and bandwidth have crucial impact on modern IoT and edge computing environments. Collecting the data is a challenging problem, as the communication is limited to existent network protocols, and adding new features requires a major infrastructure adaptation. The usage of additional monitoring protocols increases the CPU/network overhead and should be avoided as well. Therefore, we propose a two step approach for measuring latency between adjacent hops without manipulating or generating any network traffic. Inspired by audio and image compression algorithms, we developed a probabilistic method named silent consensus, where we keep the precision within a desired interval while reducing the overhead significantly. This method identifies the same packets on a sequence of network hops solely by observing the regular traffic. A linear regression helps to predict packets that are likely to appear after a fixed temporal offset based on a constrained set of historic observations. A correction of the predicted entity increases the probability for consensus between the involved hops. An extensive experimental evaluation proves that the approach delivers the expected foundation for further analysis of the network streams and the overall system.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alexandrov, A., et al.: The stratosphere platform for big data analytics. VLDB J. Int. J. Very Large Data Bases 23(6), 939–964 (2014)
Arista Networks, Inc.: arista.comwhite paperarista 7170 multi-function programmable networking. Technical report, Arista (2018). https://www.arista.com/assets/data/pdf/Whitepapers/7170_White_Paper.pdf
Bosshart, P., et al.: P4: Programming protocol-independent packet processors. ACM SIGCOMM Comput. Commun. Rev. 44(3), 87–95 (2014)
Bosshart, P., et al.: Forwarding metamorphosis: fast programmable match-action processing in hardware for sdn. In: ACM SIGCOMM Computer Communication Review, vol. 43, pp. 99–110. ACM (2013)
Brebner, G.: P4 for an FPGA target. In: P4 Workshop (2015)
Cisco: Cisco visual networking index: forecast and trends, 2017–2022. Technical report, Cisco (2018). https://www.cisco.com/c/en/us/solutions/collateral/service-provider/visual-networking-index-vni/white-paper-c11-741490.pdf
Curtis, A.R., Mogul, J.C., Tourrilhes, J., Yalagandula, P., Sharma, P., Banerjee, S.: DevoFlow: scaling flow management for high-performance networks. In: ACM SIGCOMM Computer Communication Review, vol. 41, pp. 254–265. ACM (2011)
Datta, R., Choi, S., Chowdhary, A., Park, Y.: P4Guard: designing P4 based firewall. In: MILCOM 2018–2018 IEEE Military Communications Conference (MILCOM), pp. 1–6. IEEE (2018)
Duffield, N.G., Grossglauser, M.: Trajectory sampling for direct traffic observation. IEEE/ACM Trans. Netw. (ToN) 9(3), 280–292 (2001)
Feamster, N., Rexford, J.: Why (and how) networks should run themselves. CoRR abs/1710.11583 (2017). http://arxiv.org/abs/1710.11583
Feng, Y., Wu, X., Hu, Y.: Forecasting research on the wireless mesh network throughput based on the support vector machine. Wirel. Pers. Commun. 99(1), 581–593 (2018)
Goyal, M., Guerin, R., Rajan, R.: Predicting TCP throughput from non-invasive network sampling. In: INFOCOM 2002. Proceedings of the IEEE Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies, vol. 1, pp. 180–189. IEEE (2002)
Gulenko, A., Wallschläger, M., Kao, O.: A practical implementation of in-band network telemetry in Open vSwitch. In: 2018 IEEE 7th International Conference on Cloud Networking (CloudNet), pp. 1–4. IEEE (2018)
Gupta, A., Birkner, R., Canini, M., Feamster, N., Mac-Stoker, C., Willinger, W.: Network monitoring as a streaming analytics problem. In: Proceedings of the 15th ACM Workshop on Hot Topics in Networks, pp. 106–112. ACM (2016)
Gupta, A., et al.: Sonata: query-driven network telemetry. arXiv preprint arXiv:1705.01049 (2017)
Kim, C., et al.: In-band network telemetry (INT). Technical report, The P4 Language Consortium (2016). https://p4.org/assets/INT-current-spec.pdf. Accessed 01 June 2018
Kim, C., Sivaraman, A., Katta, N., Bas, A., Dixit, A., Wobker, L.J.: In-band network telemetry via programmable dataplanes. In: ACM SIGCOMM Symposium on SDN Research (SOSR) (2015)
Liu, Z., Vorsanger, G., Braverman, V., Sekar, V.: Enabling a RISC approach for software-defined monitoring using universal streaming. In: Proceedings of the 14th ACM Workshop on Hot Topics in Networks, p. 21. ACM (2015)
Liu, Z., Bi, J., Zhou, Y., Wang, Y., Lin, Y.: NetVision: towards network telemetry as a service. In: 2018 IEEE 26th International Conference on Network Protocols (ICNP), pp. 247–248. IEEE (2018)
Lu, D., Qiao, Y., Dinda, P.A., Bustamante, F.E.: Characterizing and predicting TCP throughput on the wide area network. In: Proceedings of the 25th IEEE International Conference on Distributed Computing Systems, ICDCS 2005, pp. 414–424. IEEE (2005)
Mahalingam, M., Dutt, D.G., Duda, K., Agarwal, P.: Virtual eXtensible Local Area Network (VXLAN): a framework for overlaying virtualized layer 2 networks over layer 3 networks. RFC 7348, RFC Editor, August 2014. https://www.rfc-editor.org/rfc/rfc7348.txt
Mestre, P.: In-band OAM for IPv6. Technical report, Cisco, June 2016. https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/ipv6_nman/configuration/15-mt/ip6n-15-mt-book/ioam-ipv6.pdf
Mirza, M., Sommers, J., Barford, P., Zhu, X.: A machine learning approach to TCP throughput prediction. SIGMETRICS Perform. Eval. Rev. 35(1), 97–108 (2007). https://doi.org/10.1145/1269899.1254894
Shahbaz, M., et al.: PISCES: a programmable, protocol-independent software switch. In: SIGCOMM Conference, pp. 525–538. ACM (2016)
Suh, J., Kwon, T.T., Dixon, C., Felter, W., Carter, J.: OpenSample: a low-latency, sampling-based measurement platform for commodity SDN. In: 2014 IEEE 34th International Conference on Distributed Computing Systems (ICDCS), pp. 228–237. IEEE (2014)
Tammana, P., Agarwal, R., Lee, M.: Simplifying datacenter network debugging with pathdump. In: OSDI, pp. 233–248 (2016)
Yoon, S., Ha, T., Kim, S., Lim, H.: Scalable traffic sampling using centrality measure on software-defined networks. IEEE Commun. Mag. 55(7), 43–49 (2017)
Yu, M., Jose, L., Miao, R.: Software defined traffic measurement with opensketch. In: NSDI, vol. 13, pp. 29–42 (2013)
Zhu, Y., et al.: Packet-level telemetry in large datacenter networks. In: ACM SIGCOMM Computer Communication Review, vol. 45, pp. 479–491. ACM (2015)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Wallschläger, M., Acker, A., Kao, O. (2019). Silent Consensus: Probabilistic Packet Sampling for Lightweight Network Monitoring. In: Misra, S., et al. Computational Science and Its Applications – ICCSA 2019. ICCSA 2019. Lecture Notes in Computer Science(), vol 11619. Springer, Cham. https://doi.org/10.1007/978-3-030-24289-3_19
Download citation
DOI: https://doi.org/10.1007/978-3-030-24289-3_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-24288-6
Online ISBN: 978-3-030-24289-3
eBook Packages: Computer ScienceComputer Science (R0)