Patrick Sattler
Oliver Gasser
Ralph Holz
Abstract
Internet-wide scans are an important tool to evaluate the deployment of services. To enable large-scale application layer scans, a fast, stateless port scan (e.g., using ZMap) is often performed ahead of time to collect responsive targets. It is a common expectation that port scans on the entire IPv4 address space provide a relatively unbiased view as they cover the complete address space. Previous work, however, has found prefixes where all addresses share particular properties. In IPv6, aliased prefixes and fully responsive prefixes, i.e., prefixes where all addresses are responsive, are a well-known phenomenon. However, there is no such in-depth analysis for prefixes with these responsiveness patterns in IPv4.
This paper delves into the underlying factors of this phenomenon in the context of IPv4 and evaluates port scans on a total of 161 ports (142 TCP & 19 UDP ports) from three different vantage points. To account for packet loss and other scanning artifacts, we propose the notion of a new category of prefixes, which we call highly responsive prefixes (HRPs). Our findings show that the share of HRPs can make up 70% of responsive addresses on selected ports. Regarding specific ports, we observe that CDNs contribute to the largest fraction of HRPs on TCP/80 and TCP/443, while TCP proxies emerge as the primary cause of HRPs on other ports. Our analysis also reveals that application layer handshakes to targets outside HRPs are, depending on the chosen service, up to three times more likely to be successful compared to handshakes with targets located in HRPs. To improve future scanning campaigns conducted by the research community, we make our study's data publicly available and provide a tool for detecting HRPs. Furthermore, we propose an approach for a more efficient, ethical, and sustainable application layer target selection. We demonstrate that our approach has the potential to reduce the number of TLS handshakes by up to 75% during an Internet-wide scan while successfully obtaining 99 % of all unique certificates.
- Lance Alt, Robert Beverly, and Alberto Dainotti. 2014. Uncovering Network Tarpits with Degreaser. In Proceedings of the 30th Annual Computer Security Applications Conference (New Orleans, Louisiana, USA).Google Scholar
Digital Library
- Shehar Bano, Philipp Richter, Mobin Javed, Srikanth Sundaresan, Zakir Durumeric, Steven J. Murdoch, Richard Mortier, and Vern Paxson. 2018. Scanning the Internet for Liveness. ACM SIGCOMM Computer Communication Review (2018).Google Scholar
- Robert Beverly, William Brinkmeyer, Matthew Luckie, and Justin P. Rohrer. 2013. IPv6 Alias Resolution via Induced Fragmentation. In Proc. Passive and Active Measurement (PAM).Google Scholar
- Cloudflare. 2019. It's crowded in here! https://blog.cloudflare.com/its-crowded-in-here/Google Scholar
- Cloudflare. 2021. Unbuckling the narrow waist of IP: Addressing Agility for Names and Web Services. https://blog.cloudflare.com/addressing-agility/Google Scholar
- Cloudflare. 2023. Cloudflare Spectrum. https://www.cloudflare.com/products/cloudflare-spectrum/Google Scholar
- Cloudflare. 2023. Cloudflare Spectrum - Network ports. https://developers.cloudflare.com/fundamentals/get-started/reference/network-ports/Google Scholar
- Andrei Costin, Jonas Zaddach, Aurélien Francillon, and Davide Balzarotti. 2014. A Large-Scale Analysis of the Security of Embedded Firmwares. In 23rd USENIX Security Symposium (USENIX Security 14). USENIX Association, San Diego, CA, 95--110.Google ScholarDigital Library
- David Dittrich, Erin Kenneally, et al. 2012. The Menlo Report: Ethical principles guiding information and communication technology research. US Department of Homeland Security (2012).Google Scholar
- J. Durand, I. Pepelnjak, and G. Doering. 2015. BGP Operations and Security. RFC 7454 (Best Current Practice). https://doi.org/10.17487/RFC7454Google ScholarDigital Library
- Zakir Durumeric, David Adrian, Ariana Mirian, Michael Bailey, and J. Alex Halderman. 2015. A Search Engine Backed by Internet-Wide Scanning. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security (Denver, Colorado, USA) (CCS '15). Association for Computing Machinery, New York, NY, USA, 542--553.Google Scholar
- Zakir Durumeric, Frank Li, James Kasten, Johanna Amann, Jethro Beekman, Mathias Payer, Nicolas Weaver, David Adrian, Vern Paxson, Michael Bailey, and J. Alex Halderman. 2014. The Matter of Heartbleed. In Proceedings of the 2014 Conference on Internet Measurement Conference (Vancouver, BC, Canada) (IMC '14). Association for Computing Machinery, New York, NY, USA, 475--488.Google ScholarDigital Library
- Zakir Durumeric, Eric Wustrow, and J. Alex Halderman. 2013. ZMap: Fast Internet-wide Scanning and Its Security Applications. In Proc. USENIX Security Symposium. Washington, D.C., USA.Google Scholar
- Marwan Fayed, Lorenz Bauer, Vasileios Giotsas, Sami Kerola, Marek Majkowski, Pavel Odintsov, Jakub Sitnicki, Taejoong Chung, Dave Levin, Alan Mislove, Christopher A. Wood, and Nick Sullivan. 2021. The Ties That Un-Bind: Decoupling IP from Web Services and Sockets for Robust Addressing Agility at CDN-Scale. In Proceedings of the 2021 ACM SIGCOMM 2021 Conference (SIGCOMM '21). Association for Computing Machinery, New York, NY, USA.Google ScholarDigital Library
- Oliver Gasser, Benjamin Hof, Max Helm, Maciej Korczynski, Ralph Holz, and Georg Carle. 2018. In Log We Trust: Revealing Poor Security Practices with Certificate Transparency Logs and Internet Measurements. In Passive and Active Measurement Conference 2018.Google Scholar
- Oliver Gasser, Quirin Scheitle, Pawel Foremski, Qasim Lone, Maciej Korczynski, Stephen D. Strowes, Luuk Hendriks, and Georg Carle. 2018. Clusters in the Expanse: Understanding and Unbiasing IPv6 Hitlists. In Proc. ACM Int. Measurement Conference (IMC) (Boston, MA, USA).Google ScholarDigital Library
- Oliver Gasser, Quirin Scheitle, Sebastian Gebhard, and Georg Carle. 2016. Scanning the IPv6 Internet: Towards a Comprehensive Hitlist. In Proc. 8th Int. Workshop on Traffic Monitoring and Analysis. Louvain-la-Neuve, Belgium.Google Scholar
- Oliver Gasser, Markus Sosnowski, Patrick Sattler, and Johannes Zirngibl. 2023. Goscanner. Retrieved 2023-03--24 from https://github.com/tumi8/goscannerGoogle Scholar
- Robert Graham. [n. d.]. MASSCAN: Mass IP port scanner. https://github.com/robertdavidgraham/masscanGoogle Scholar
- Marcia Hofmann. 2013. Legal Considerations for Widespread Scanning. Retrieved 2023-09--26 from https://www.rapid7.com/blog/post/2013/10/30/legal-considerations-for-widespread-scanning/Google Scholar
- https://csirt.divd.nl/. 2023. Making the internet safer through Coordinated Vulnerability Disclosure. Retrieved 2023-03--24 from https://csirt.divd.nl/Google Scholar
- ICANN. 2023. CZDS - Centralized Zone Data Service. Retrieved 2023--10-05 from https://czds.icann.org/Google Scholar
- Liz Izhikevich, Renata Teixeira, and Zakir Durumeric. 2021. LZR: Identifying Unexpected Internet Services. In Proc. USENIX Security Symposium. https://www.usenix.org/conference/usenixsecurity21/presentation/izhikevichGoogle Scholar
- Liz Izhikevich, Renata Teixeira, and Zakir Durumeric. 2022. Predicting IPv4 Services across All Ports. In Proceedings of the ACM SIGCOMM 2022 Conference (Amsterdam, Netherlands) (SIGCOMM '22). Association for Computing Machinery, New York, NY, USA, 503--515. https://doi.org/10.1145/3544216.3544249Google ScholarDigital Library
- Johannes Klick, Stephan Lau, Matthias Wählisch, and Volker Roth. 2016. Towards Better Internet Citizenship: Reducing the Footprint of Internet-Wide Scans by Topology Aware Prefix Selection. In Proc. ACM Int. Measurement Conference (IMC) (Santa Monica, California, USA). Association for Computing Machinery, New York, NY, USA.Google ScholarDigital Library
- Matthew Luckie, Robert Beverly, William Brinkmeyer, and kc claffy. 2013. Speedtrap: Internet-Scale IPv6 Alias Resolution. In Proc. ACM Int. Measurement Conference (IMC) (Barcelona, Spain).Google Scholar
- MANRS. 2021. Prefix filter configuration tools. https://www.manrs.org/isps/guide/filtering/Google Scholar
- Austin Murdock, Frank Li, Paul Bramsen, Zakir Durumeric, and Vern Paxson. 2017. Target Generation for Internet-Wide IPv6 Scanning. In Proc. ACM Int. Measurement Conference (IMC) (London, United Kingdom).Google ScholarDigital Library
- University of Oregon. 2023. University of Oregon Route Views Project. http://www.routeviews.org/routeviews/Google Scholar
- Ramakrishna Padmanabhan, Zhihao Li, Dave Levin, and Neil Spring. 2015. UAv6: Alias Resolution in IPv6 Using Unused Addresses. In Proc. Passive and Active Measurement (PAM).Google ScholarCross Ref
- Craig Partridge and Mark Allman. 2016. Addressing Ethical Considerations in Network Measurement Papers. Commun. ACM 59, 10 (Oct. 2016).Google ScholarDigital Library
- The ZMap Project. 2023. ZGrab 2.0. Retrieved 2023-03--24 from https://github.com/zmap/zgrab2Google Scholar
- Patrick Sattler, Johannes Zirngibl, Mattijs Jonker, Oliver Gasser, Georg Carle, and Ralph Holz. 2023. Data and Analysis at TUM University Library. https://mediatum.ub.tum.de/1723389 doi:10.14459/2023mp1723389.Google ScholarCross Ref
- Patrick Sattler, Johannes Zirngibl, Mattijs Jonker, Oliver Gasser, Georg Carle, and Ralph Holz. 2023. HRP Website with data. Retrieved 2023--10-05 from https://hrp-stats.github.io/Google Scholar
- Khwaja Zubair Sediqi, Lars Prehn, and Oliver Gasser. 2022. Hyper-Specific Prefixes: Gotta Enjoy the Little Things in Interdomain Routing. ACM SIGCOMM Computer Communication Review 52 (June 2022). Issue 2. https://doi.org/10.1145/3544912.3544916Google ScholarDigital Library
- Shadowserver. 2023. Shadowserver - Lighting the way to a more secure Internet. Retrieved 2023-03--24 from https://www.shadowserver.org/Google Scholar
- Shodan. 2023. Shodan Dashboard. Retrieved 2023-03--24 from https://www.shodan.io/dashboardGoogle Scholar
- Rapid7 Project Sonar. 2023. Open Data. Retrieved 2023-03--24 from https://opendata.rapid7.com/Google Scholar
- Roland van Rijswijk-Deij, Mattijs Jonker, Anna Sperotto, and Aiko Pras. 2016. A high-performance, scalable infrastructure for large-scale active DNS measurements. IEEE journal on selected areas in communications 34, 6 (2016), 1877--1888.Google ScholarCross Ref
- Gerry Wan, Liz Izhikevich, David Adrian, Katsunari Yoshioka, Ralph Holz, Christian Rossow, and Zakir Durumeric. 2020. On the Origin of Scanning: The Impact of Location on Internet-Wide Scans. In Proceedings of the ACM Internet Measurement Conference (Virtual Event, USA) (IMC '20). Association for Computing Machinery, New York, NY, USA, 662--679. https://doi.org/10.1145/3419394.3424214Google ScholarDigital Library
- Johannes Zirngibl, Steffen Deusch, Patrick Sattler, Juliane Aulbach, Georg Carle, and Mattijs Jonker. 2022. Domain Parking: Largely Present, Rarely Considered!. In Proc. Network Traffic Measurement and Analysis Conference (TMA) 2022.Google Scholar
- Johannes Zirngibl, Patrick Sattler, and Georg Carle. 2023. A First Look at SVCB and HTTPS DNS Resource Records in the Wild. In 2023 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW). 470--474.Google ScholarCross Ref
- Johannes Zirngibl, Lion Steger, Patrick Sattler, Oliver Gasser, and Georg Carle. 2022. Rusty Clusters? Dusting an IPv6 Research Foundation. In Proc. ACM Int. Measurement Conference (IMC) (Nice, France).Google ScholarDigital Library
Index Terms
- Packed to the Brim: Investigating the Impact of Highly Responsive Prefixes on Internet-wide Measurement Campaigns
Recommendations
IPREDS: Efficient Prediction System for Internet-wide Port and Service Scanning
PACMNETInternet-wide port and service scanning, a vital tool for network research, is unaffordable in time and network bandwidth consumption. However, scanning only a portion of ports and services may lead to erroneous research conclusions. Previous work has ...
An internet-wide view of internet-wide scanning
SEC'14: Proceedings of the 23rd USENIX conference on Security SymposiumWhile it is widely known that port scanning is widespread, neither the scanning landscape nor the defensive reactions of network operators have been measured at Internet scale. In this work, we analyze data from a large network telescope to study ...
Design and Implement of Common Network Security Scanning System
IUCE '09: Proceedings of the 2009 International Symposium on Intelligent Ubiquitous Computing and EducationThis paper discusses the network security scanning and some scanning methods which contain port scanning, vulnerability scanning and remote operating system detection are studied. In order to reduce the complexity and get high performance, the ...
Comments