Challenges in the capture and dissemination of measurements from high-speed networks

R.G. Clegg; M.S. Withall; A.W. Moore; I.W. Phillips; D.J. Parish; M. Rio; R. Landa; H. Haddadi; K. Kyriakopoulos; J. Augé; R. Clayton; D. Salmon

Challenges in the capture and dissemination of measurements from high-speed networks

Access Full Text

Challenges in the capture and dissemination of measurements from high-speed networks

Author(s): R.G. Clegg ; M.S. Withall ; A.W. Moore ; I.W. Phillips ; D.J. Parish ; M. Rio ; R. Landa ; H. Haddadi ; K. Kyriakopoulos ; J. Augé ; R. Clayton ; D. Salmon
DOI: 10.1049/iet-com.2008.0068

For access to this article, please select a purchase option:

Buy article PDF

Buy Knowledge Pack

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership

Recommend Title Publication to library

IET Communications — Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

Author(s): R.G. Clegg ¹ ; M.S. Withall ² ; A.W. Moore ³ ; I.W. Phillips ⁴ ; D.J. Parish ² ; M. Rio ¹ ; R. Landa ¹ ; H. Haddadi ¹ ; K. Kyriakopoulos ² ; J. Augé ³ ; R. Clayton ³ ; D. Salmon ⁵
- Affiliations: 1: Department of Electronic and Electrical Engineering, University College London, London, UK
  2: Department of Electronic and Electrical Engineering, Loughborough University, Loughborough, UK
  3: Computer Laboratory, University of Cambridge, Cambridge, UK
  4: Department of Computer Science, Department of Electronic and Electrical Engineering, Loughborough University, Loughborough, UK
  5: Department of Computer Science, Department of Electronic and Electrical Engineering, JANET (UK), Didcot, UK
Source: Volume 3, Issue 6, June 2009, p. 957 – 966
DOI: 10.1049/iet-com.2008.0068 , Print ISSN 1751-8628, Online ISSN 1751-8636

Published

The production of a large-scale monitoring system for a high-speed network leads to a number of challenges. These challenges are not purely technical but also socio-political and legal. The number of stakeholders in such monitoring activity is large including the network operators, the users, the equipment manufacturers and, of course, the monitoring researchers. The MASTS project (measurement at all scales in time and space) was created to instrument the high-speed JANET Lightpath network and has been extended to incorporate other paths supported by JANET(UK). Challenges the project has faced included: simple access to the network; legal issues involved in the storage and dissemination of the captured information, which may be personal; the volume of data captured and the rate at which these data appear at store. To this end, the MASTS system will have established four monitoring points each capturing packets on a high-speed link. Traffic header data will be continuously collected, anonymised, indexed, stored and made available to the research community. A legal framework for the capture and storage of network measurement data has been developed which allows the anonymised IP traces to be used for research purposes.

References

1. 1)
  - W.E. Leland , M.S. Taqqu , W. Willinger , D.V. Wilson . On the self-similar nature of Ethernet traffic (extended version). IEEE/ACM Trans. Netw. , 1 - 15
2. 2)
  - Directive 2002/58/EC of the European Parliament and of the Council of 12 July 2002 concerning the processing of personal data and the protection of privacy in the electronic communications sector (Directive on privacy and electronic communications. Official J. L , 0037 - 0047
3. 3)
  - BT; ‘Delivering the future, BT's 21st century network’, http://www.btplc.com/21CN/.
4. 4)
  - Soltis, S.R., Ruwart, T.M., O'Keefe, M.T.: `The global file system', Fifth NASA Goddard Space Flight Center Conf. Mass Storage Systems and Technologies, September 1996, MD, 46, College Park, p. 24–31.
5. 5)
  - ‘gzip website’, http://www.gzip.org.
6. 6)
  - ‘GFS project page’, http://sources.redhat.com/cluster/gfs/.
7. 7)
  - ‘NLANR website’, http://pma.nlanr.net/.
8. 8)
  - ‘CRAWDAD website’, http://crawdad.cs.dartmouth.edu/.
9. 9)
  - C.D. Keeling . Rewards and penalties of monitoring the earth. Ann. Rev. Energy Environ. , 25 - 82
10. 10)
  - Fox, G., Walker, D.: `e-science gap analysis', Technical report, June 2003, http://grids.ucs.indiana.edu/ptliupages/publications/GapAnalysis30June03v2.pdf.
11. 11)
  - Fisk, M., Varghese, G.: `Agile and scalable analysis of network events', Proc. SIGCOMM Internet Measurement Workshop, November 2002, ACM.
12. 12)
  - Kyriakopoulos, K., Parish, D.: `A live system for wavelet compression of high speed computer network measurements', Proc. Passive and Active Measurement (PAM) 2007, April 2007, p. 241–244.
13. 13)
  - Her Majesty's Stationery Office: ‘Regulation of investigatory powers act. chapter 23’, http://www.opsi.gov.uk/acts/acts2000/ukpga_20000023_en_1, 2000.
14. 14)
  - Donnelly, S.: `High precision timing in passive measurements of data networks', June 2002, PhD, University of Waikato.
15. 15)
  - ‘LZO website’, http://www.oberhumer.com/opensource/lzo/.
16. 16)
  - M. Withall , I. Phillips , D. Parish . Network visualization: a review. IET Commun. , 365 - 372
17. 17)
  - Zhou, X., Mieghem, P.V.: `Hopcount and E2E delay: IPv6 versus IPv4', Proc. Passive and Active Measurement (PAM) 2005, March–April 2005, p. 345–348.
18. 18)
  - Paxson, V.: `Strategies for sound internet measurement', IMC ‘04: Proc. 4th ACM/SIGCOMM Conf. Internet Measurement, 2004, ACM, New York, NY, USA, p. 263–271.
19. 19)
  - Directive 95/46/EC of the European Parliament and of the Council of 24 October 1995 on the protection of individuals with regard to the processing of personal data and on the free movement of such data. Official J. L , 0031 - 0050
20. 20)
  - ‘WAND website’, http://research.wand.net.nz/software/libtrace.php.
21. 21)
  - K. Claffy . Measuring the internet. IEEE Internet Comput. , 1 , 73 - 75
22. 22)
  - ‘Postgresql website’, http://www.postgresql.org/.
23. 23)
  - ‘LINX website’, https://www.linx.net/.
24. 24)
  - Her Majesty's Stationery Office: ‘Data protection Act. Chapter 29’, http://www.opsi.gov.uk/Acts/Acts1998/ukpga_19980029_en_1, 1998.
25. 25)
  - J. Fan , J. Xu , M.H. Ammar , S.B. Moon . Prefix-preserving IP address anonymization: measurement-based security evaluation and a new cryptography-based scheme. Comput. Netw. , 2 , 253 - 272
26. 26)
  - Zhang, J., Moore, A.: `Traffic trace artifacts due to monitoring via port mirroring', Workshop on End-to-End Monitoring Techniques and Services, 2007. E2EMON ’07, May 2007, Munich, Germany, p. 1–8.
27. 27)
  - D.A. Patterson , D.D. Clark , A. Karlin . (2001) Looking over the fence at networks: a neighbor's view of networking research.
28. 28)
  - Murray, M., Claffy, K.: `Measuring the immeasurable: global internet measurement infrastructure', Proc. Passive and Active Measurement (PAM) 2001, April 2001, p. 9.
29. 29)
  - O. Bashir , I. Phillips , D. Parish , J. Adams , T. Spencer . The management and processing of network performance information. BT Technol. J. , 203 - 212
30. 30)
  - ‘Turbogears website’, http://www.turbogears.org/.
31. 31)
  - CISCO: ‘Catalyst switched port analyzer (SPAN) configuration example’, http://www.cisco.com/warp/public/473/41.html.
32. 32)
  - MASTS Consortium; ‘MASTS website’, http://www.mastsproject.org/.
33. 33)
  - E.L. Cashin . Kernel korner: ATA over Ethernet: putting hard drives on the LAN. Linux J. , 134
34. 34)
  - ‘TCPDUMP website’, http://www.tcpdump.org/.
35. 35)
  - ‘Wireshark website’, http://www.wireshark.org/.
36. 36)
  - ‘RIPE website', http://www.ripe.net/ttm/.

Login

Not registered yet?

Share

Tools

Login to add to favourites

Key

Challenges in the capture and dissemination of measurements from high-speed networks

Challenges in the capture and dissemination of measurements from high-speed networks

Buy article PDF

Buy Knowledge Pack

Thank you

References

Related content