Abstract
Time matters. In a networked world, we would like mobile devices to provide a crisp user experience and applications to instantaneously return results. Unfortunately, application performance does not depend solely on processing time, but also on a number of different components that are commonly counted in the overall system latency. Latency is more than just a nuisance to the user, poorly accounted-for, it degrades application performance. In fields such as high frequency trading, as well as in many data centers, latency translates easily to financial losses. Research to date has focused on specific contributions to latency: from improving latency within the network to latency control on the application level. This paper takes an holistic approach to latency, and aims to provide a break-down of end-to-end latency from the application level to the wire. Using a set of crafted experiments, we explore the many contributors to latency. We assert that more attention should be paid to the latency within the host, and show that there is no silver bullet to solve the end-to-end latency challenge in data centers. We believe that a better understanding of the key elements influencing data center latency can trigger a more focused research, improving the user’s quality of experience.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
- 2.
- 3.
- 4.
Our latency-injection appliance is an open-source contributed project as part of NetFPGA SUME since release 1.4.0.
- 5.
The maximum latency introduced is a function of the configured line-rate. The appliance can add up to 700 \(\upmu \mathrm {s}\) of latency at full 10 Gb/s rate, and up to 7 \(\mathrm {s}\) at 100 Mbps.
- 6.
We note that the resolution of the DAG of 7.5 \(\mathrm {ns}\) puts short fiber measurements within this range of error.
- 7.
The source code for the test is provided with the NIC, but is not open source.
- 8.
Based on evaluation on Xeon E5-2637 v3, i7-6700K and i7-4770 based platforms, and Linux kernels ranging from 3.18.42 to 4.4.0-42.
References
Barroso, L.A.: Landheld Computing. In: IEEE International Solid State Circuits Conference (ISSCC) (2014). Keynote
Cheshire, S.: It’s the latency, stupid. http://www.stuartcheshire.org/rants/Latency.html. Accessed July 2016
Guo, C., et al.: RDMA over commodity ethernet at scale. In: SIGCOMM 2016 (2016)
Hemminger, S.: NetEm - Network Emulator. http://man7.org/linux/man-pages/man8/tc-netem.8.html. Accessed July 2016
Kalia, A., et al.: Design guidelines for high performance RDMA systems. In: USENIX ATC, vol. 16, pp. 437–450 (2016)
Mayer, M.: What Google knows. In: Web 2.0 Summit (2006)
Mittal, R., et al.: TIMELY: RTT-based congestion control for the datacenter. SIGCOMM Comput. Commun. Rev. 45, 537–550 (2015). ACM
Nussbaum, L., Richard, O.: A comparative study of network link emulators. In: SpringSim 2009, pp. 85:1–85:8 (2009)
Oracle: Oracle VM VirtualBox. https://www.virtualbox.org/. Accessed Oct 2016
Paoloni, G.: How to benchmark code execution times on Intel IA-32 and IA-64 instruction set architectures. Technical report 324264–001, Intel (2010)
Patterson, D.A.: Latency lags bandwidth. Commun. ACM 47(10), 71–75 (2004)
Rumble, S.M., et al.: It’s time for low latency. In: HotOS 2013, p. 11. USENIX Association (2011)
SAP: Big data and smart trading (2012)
Singh, A., et al.: Jupiter rising: a decade of clos topologies and centralized control in Google’s datacenter network. SIGCOMM Comput. Commun. Rev. 45(4), 183–197 (2015)
Tolly Enterprises: Mellanox spectrum vs. broadcom StrataXGS Tomahawk 25GbE & 100GbE performance evaluation - evaluating consistency & predictability. Technical report 216112 (2016)
Zilberman, N., et al.: NetFPGA SUME: toward 100 Gbps as research commodity. IEEE Micro 34(5), 32–41 (2014)
Acknowledgments
We would like to thank the many people who contributed to this paper. We would like to thank Salvator Galea and Robert N Watson, who contributed to early work on this paper. This work has received funding from the EPSRC grant EP/K034723/1, Leverhulme Trust Early Career Fellowship ECF-2016-289, European Union’s Horizon 2020 research and innovation programme 2014-2018 under the SSICLOPS (grant agreement No. 644866), ENDEAVOUR (grant agreement No. 644960) and EU FP7 Marie Curie ITN METRICS (grant agreement No. 607728).
Dataset. A reproduction environment of the experiments, and the experimental results, are both available at http://www.cl.cam.ac.uk/research/srg/netos/projects/latency/pam2017/ and https://doi.org/10.17863/CAM.7418.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Zilberman, N. et al. (2017). Where Has My Time Gone?. In: Kaafar, M., Uhlig, S., Amann, J. (eds) Passive and Active Measurement. PAM 2017. Lecture Notes in Computer Science(), vol 10176. Springer, Cham. https://doi.org/10.1007/978-3-319-54328-4_15
Download citation
DOI: https://doi.org/10.1007/978-3-319-54328-4_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-54327-7
Online ISBN: 978-3-319-54328-4
eBook Packages: Computer ScienceComputer Science (R0)