Leigh Metcalf
ABSTRACT
Motivation: We compare the contents of 86 Internet blacklists to provide a view of the whole ecosystem of blocking network touch points and blacklists. We aim to formalize and evaluate practitioner tacit knowledge of the fatigue of playing "whack-a-mole" against resilient adversary resources. Method: Lists are compared to lists of the same data type (domain name or IP address). Different phases of the study use different comparisons. Comparisons include how many lists an indicator is unique to; list sizes; expanded list characterization and intersection; pairwise intersections of all lists; and following, a statistical test we define to determine if one list adds elements shortly after another. Results: Based on a synthesis of multiple methods, domain-name-based indicators are unique to one list 96.16% to 97.37% of the time. IP-address-based indicators are unique to one list 82.46% to 95.24% of the time. Discussion: There is little overlap between blacklists. Though there are exceptions, the intersection between lists remains low even after expanding each list to a larger neighborhood of related indicators. Few lists consistently provide content before other lists if there is intersection. These results suggest that each blacklist describes a distinct sort of malicious activity and that even merging all lists there is no global ground truth to acquire. Practical insights include (1) network defenders are advised to obtain and evaluate as many lists as practical, (2) "whack-a-mole" is inevitable due to list dynamics, barring a strategic change, an (3) academics comparing their results to one or a few blacklists to test accuracy are advised to reconsider this validation technique.
- 2015 data breach investigations report (DBIR). Tech. rep., Verizon, 2015.Google Scholar
- Burger, E. W., Goodman, M. D., Kampanakis, P., and Zhu, K. A. Taxonomy model for cyber threat intelligence information exchange technologies. In Proceedings of the 2014 ACM Workshop on Information Sharing & Collaborative Security (2014), ACM, pp. 51--60. Google ScholarDigital Library
- CERT/NetSA at Carnegie Mellon University. CERT/CC Route Views Project Page. {Accessed: Feb 13, 2014}.Google Scholar
- Collins, M. Using vantage to manage complex sensor networks. In FloCon 2015, 11th Annual (Portland, OR, January 2015), Software Engineering Institute, Carnegie Mellon University.Google Scholar
- Google. Google Safe Browsing FAQ. http://code.google.com/apis/safebrowsing/safebrowsing_faq.html, November 2, 2011.Google Scholar
- Grier, C., Ballard, L., Caballero, J., Chachra, N., Dietrich, C. J., Levchenko, K., Mavrommatis, P., McCoy, D., Nappa, A., Pitsillidis, A., Provos, N., Rafique, M. Z., Rajab, M. A., Rossow, C., Thomas, K., Paxson, V., Savage, S., and Voelker, G. M. Manufacturing compromise: The emergence of exploit-as-a-service. In Proceedings of the 2012 ACM Conference on Computer and Communications Security (Raleigh, North Carolina, USA, 2012), CCS '12, ACM, pp. 821--832. Google ScholarDigital Library
- Hallenbeck, C., King, C., Spring, J. M., and Vixie, P. Abuse of customer premise equipment and recommended actions. In Black Hat USA 2014 (Las Vegas, Nevada, Aug 7, 2014), UBM.Google Scholar
- Hatleback, E., and Spring, J. M. Exploring a mechanistic approach to experimentation in computing. Philosophy & Technology 27, 3 (2014), 441--459.Google ScholarCross Ref
- Kührer, M., Rossow, C., and Holz, T. Paint it black: Evaluating the effectiveness of malware blacklists. Tech. Rep. TR-HGI-2014-002, Ruhr-Universitat Bochum, Horst Görtz Institute for IT Security, June 2014.Google ScholarCross Ref
- Metcalf, L. B., and Spring, J. M. Everything you wanted to know about blacklists but were afraid to ask. Tech. Rep. CERTCC-2013--39, Software Engineering Institute, CERT Coordination Center, Pittsburgh, PA, 2013.Google Scholar
- Metcalf, L. B., and Spring, J. M. Blacklist ecosystem analysis update: 2014. Tech. Rep. CERTCC-2014--82, Software Engineering Institute, CERT Coordination Center, Pittsburgh, PA, December 2014.Google Scholar
- RIPE Network Coordination Center. Routing information service (RIS). http://www.ripe.net/data-tools/stats/ris/routing-information-service, January 3, 2012.Google Scholar
- Route-Views. University of oregon route views project. http://www.routeviews.org, January 3, 2012.Google Scholar
- Scheper, C., Cantor, S., and Maughan, D. Predict: a trusted framework for sharing data for cyber security research. In Proceedings of the First Workshop on Building Analysis Datasets and Gathering Experience Returns for Security (2011), ACM, pp. 105--106. Google ScholarDigital Library
- Serrano, O., Dandurand, L., and Brown, S. On the design of a cyber security data sharing system. In Proceedings of the 2014 ACM Workshop on Information Sharing & Collaborative Security (2014), ACM, pp. 61--69. Google ScholarDigital Library
- Sharma, V., Bartlett, G., and Mirkovic, J. Critter: Content-rich traffic trace repository. In Proceedings of the 2014 ACM Workshop on Information Sharing & Collaborative Security (2014), ACM, pp. 13--20. Google ScholarDigital Library
- Sinervo, P. K. Signal significance in particle physics. In Advanced statistical techniques in particle physics (Durham, UK, 2002), M. R. Whalley and L. Lyons, Eds.Google Scholar
- Sinha, S., Bailey, M., and Jahanian, F. Shades of grey: On the effectiveness of reputation-based "blacklists". In Malicious and Unwanted Software, 2008. MALWARE 2008. 3rd International Conference on (2008), IEEE, pp. 57--64.Google ScholarCross Ref
- Sood, A. K., and Enbody, R. J. Crimeware-as-a-service: A survey of commoditized crimeware in the underground market. International Journal of Critical Infrastructure Protection 6, 1 (2013), 28--38.Google ScholarCross Ref
- Spring, J. M. Modeling malicious domain name take-down dynamics: Why eCrime pays. In IEEE eCrime Researchers Summit (September 17, 2013), Anti-Phishing Working Group.Google Scholar
- Spring, J. M. A notation for describing the steps in indicator expansion. In IEEE eCrime Researchers Summit (September 17, 2013), Anti-Phishing Working Group.Google Scholar
- Spring, J. M. Toward realistic modeling criteria of games in internet security. Journal of Cyber Security & Information Systems 2, 2 (2014), 2--11.Google Scholar
- Spring, J. M., Kern, S., and Summers, A. Global adversarial capability modeling. In IEEE eCrime Researchers Summit (Barcelona, May 28, 2015), Anti-Phishing Working Group, pp. 22--42.Google Scholar
- Sundaramurthy, S. C., McHugh, J., Ou, X. S., Rajagopalan, S. R., and Wesch, M. An anthropological approach to studying csirts. IEEE Security & Privacy, 5 (2014), 52--60.Google ScholarCross Ref
- Thomas, M., Metcalf, L., Spring, J. M., Krystosek, P., and Prevost, K. Silk: A tool suite for unsampled network flow analysis at scale. In IEEE BigData Congress (Anchorage, AK, July 2014), IEEE. Google ScholarDigital Library
- Trost, R. Threat intelligence library - a new revolutionary technology to enhance the soc battle rhythm! In Black Hat USA 2014 (Las Vegas, Nevada, Aug 7, 2014), UBM.Google Scholar
- Zhang, J., Chivukula, A., Bailey, M., Karir, M., and Liu, M. Characterization of blacklists and tainted network traffic. In Passive and Active Measurement, M. Roughan and R. Chang, Eds., vol. 7799 of Lecture Notes in Computer Science. Springer Berlin Heidelberg, 2013, pp. 218--228. Google ScholarDigital Library
Index Terms
- Blacklist Ecosystem Analysis: Spanning Jan 2012 to Jun 2014
Recommendations
Gossip: Automatically Identifying Malicious Domains from Mailing List Discussions
ASIA CCS '17: Proceedings of the 2017 ACM on Asia Conference on Computer and Communications SecurityDomain names play a critical role in cybercrime, because they identify hosts that serve malicious content (such as malware, Trojan binaries, or malicious scripts), operate as command-and-control servers, or carry out some other role in the malicious ...
Filtering spam with behavioral blacklisting
CCS '07: Proceedings of the 14th ACM conference on Computer and communications securitySpam filters often use the reputation of an IP address (or IP address range) to classify email senders. This approach worked well when most spam originated from senders with fixed IP addresses, but spam today is also sent from IP addresses for which ...
Modeling and Analysis of Active Benign Worms and Hybrid Benign Worms Containing the Spread of Worms
ICN '07: Proceedings of the Sixth International Conference on NetworkingWorms are a serious and growing threat to network and traditional antivirus technologies do not currently scale to deal with the worm threat. Benign worms, especially active benign worms and hybrid benign worms, become a new active countermeasure. In ...
Comments