Owen S. Hofmann
Emmett Witchel
ABSTRACT
Kernel rootkits that modify operating system state to avoid detection are a dangerous threat to system security. This paper presents OSck, a system that discovers kernel rootkits by detecting malicious modifications to operating system data. OSck integrates and extends existing techniques for detecting rootkits, and verifies safety properties for large portions of the kernel heap with minimal overhead. We deduce type information for verification by analyzing unmodified kernel source code and in-memory kernel data structures.
High-performance integrity checks that execute concurrently with a running operating system create data races, and we demonstrate a deterministic solution for ensuring kernel memory is in a consistent state. We introduce two new classes of kernel rootkits that are undetectable by current systems, motivating the need for the OSck API that allows kernel developers to conveniently specify arbitrary integrity properties.
- M. Abadi, M. Budiu, U. Erlingsson, and J. Ligatti. Control-flow integrity. In Proceedings of the 12th ACM Conference on Computer and Communications Security, CCS '05, pages 340--353, New York, NY, USA, 2005. ACM. Google Scholar
Digital Library
- Aleph One. Smashing the stack for fun and profit. Phrack, 7(49), Nov 1996.Google Scholar
- W. A. Arbaugh, D. J. Farber, and J. M. Smith. A secure and reliable bootstrap architecture. In SP '97: Proceedings of the 1997 IEEE Symposium on Security and Privacy, page 65, Washington, DC, USA, 1997. IEEE Computer Society. Google ScholarDigital Library
- A. Baliga, P. Kamat, and L. Iftode. Lurking in the shadows: Identifying systemic threats to kernel data. In Proceedings of the 2007 IEEE Symposium on Security and Privacy, SP '07, pages 246--251, Washington, DC, USA, 2007. IEEE Computer Society. Google ScholarDigital Library
- P. Barham, B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, R. Neugebauer, I. Pratt, and A. Warfield. Xen and the art of virtualization. In Proceedings of the nineteenth ACM Symposium on Operating Systems Principles, SOSP '03, pages 164--177, New York, NY, USA, 2003. ACM. Google ScholarDigital Library
- C. Bienia. Benchmarking Modern Multiprocessors. PhD thesis, Princeton University, January 2011. Google ScholarDigital Library
- J. Bonwick. The slab allocator: an object-caching kernel memory allocator. In Proceedings of the USENIX Summer 1994 Technical Conference, pages 6--6, Berkeley, CA, USA, 1994. USENIX Association. Google ScholarDigital Library
- S. Chen, J. Xu, E. C. Sezer, P. Gauriar, and R. K. Iyer. Non-control-data attacks are realistic threats. In Proceedings of the 14th conference on USENIX Security Symposium - Volume 14, pages 12--12, Berkeley, CA, USA, 2005. USENIX Association. Google ScholarDigital Library
- X. Chen, T. Garfinkel, E. C. Lewis, P. Subrahmanyam, C. A. Waldspurger, D. Boneh, J. Dwoskin, and D. R. Ports. Overshadow: a virtualization-based approach to retrofitting protection in commodity operating systems. In Proceedings of the 13th international conference on Architectural support for programming languages and operating systems, ASPLOS XIII, pages 2--13, New York, NY, USA, 2008. ACM. Google ScholarDigital Library
- C. Cowan, C. Pu, D. Maier, H. Hintony, J. Walpole, P. Bakke, S. Beattie, A. Grier, P. Wagle, and Q. Zhang. Stackguard: automatic adaptive detection and prevention of buffer-overflow attacks. In Proceedings of the 7th conference on USENIX Security Symposium - Volume 7, pages 5--5, Berkeley, CA, USA, 1998. USENIX Association. Google ScholarDigital Library
- A. Cozzie, F. Stratton, H. Xue, and S. T. King. Digging for data structures. In Proceedings of the 8th USENIX conference on Operating systems design and implementation, OSDI'08, pages 255--266, Berkeley, CA, USA, 2008. USENIX Association. Google ScholarDigital Library
- T. Garfinkel, B. Pfaff, J. Chow, M. Rosenblum, and D. Boneh. Terra: a virtual machine-based platform for trusted computing. In Proceedings of the nineteenth ACM Symposium on Operating Systems Principles, SOSP '03, pages 193--206, New York, NY, USA, 2003. ACM. Google ScholarDigital Library
- T. Garfinkel and M. Rosenblum. A virtual machine introspection based architecture for intrusion detection. In Procedings of the Network and Distributed Systems Security Symposium, pages 191--206, 2003.Google Scholar
- J. L. Henning. SPEC CPU2006 benchmark descriptions. SIGARCH Comput. Archit. News, 34(4):1--17, 2006. Google ScholarDigital Library
- G. Hoglund and J. Butler. Rootkits: Subverting the Windows Kernel. Addison-Wesley Professional, 2005. Google ScholarDigital Library
- J. H. Howard, M. L. Kazar, S. G. Menees, D. A. Nichols, M. Satyanarayanan, R. N. Sidebotham, and M. J. West. Scale and performance in a distributed file system. ACM Transactions on Computer Systems, 6:51--81, February 1988. Google ScholarDigital Library
- R. Hund, T. Holz, and F. C. Freiling. Return-oriented rootkits: bypassing kernel code integrity protection mechanisms. In Proceedings of the 18th conference on USENIX security symposium, SSYM'09, pages 383--398, Berkeley, CA, USA, 2009. USENIX Association. Google ScholarDigital Library
- D. X. Junghwan Rhee. LiveDM: Temporal mapping of dynamic kernel memory for dynamic kernel malware analysis and debugging. Technical report, 2 2010.Google Scholar
- B. Kauer. OSLO: improving the security of trusted computing. In Proceedings of 16th USENIX Security Symposium on USENIX Security Symposium, pages 16:1--16:9, Berkeley, CA, USA, 2007. USENIX Association. Google ScholarDigital Library
- A. Kivity. kvm: The Linux Virtual Machine Monitor. In The 2007 Ottawa Linux Symposium, OLS '07, pages 225--230, July 2007.Google Scholar
- P. A. Loscocco, P. W. Wilson, J. A. Pendergrass, and C. D. McDonell. Linux kernel integrity measurement using contextual inspection. In Proceedings of the 2007 ACM workshop on Scalable trusted computing, STC '07, pages 21--29, New York, NY, USA, 2007. ACM. Google ScholarDigital Library
- G. C. Necula, S. McPeak, S. P. Rahul, and W. Weimer. Cil: Intermediate language and tools for analysis and transformation of c programs. In Proceedings of the 11th International Conference on Compiler Construction, CC '02, pages 213--228, London, UK, 2002. Springer-Verlag. Google ScholarDigital Library
- N. L. Petroni, Jr. and M. Hicks. Automated detection of persistent kernel control-flow attacks. In Proceedings of the 14th ACM Conference on Computer and Communications Security, CCS '07, pages 103--115, New York, NY, USA, 2007. ACM. Google ScholarDigital Library
- D. E. Porter, O. S. Hofmann, C. J. Rossbach, A. Benn, and E. Witchel. Operating system transactions. In Proceedings of the ACM SIGOPS 22nd Symposium on Operating Systems Principles, SOSP '09, pages 161--176, New York, NY, USA, 2009. ACM. Google ScholarDigital Library
- J. Rhee, R. Riley, D. Xu, and X. Jiang. Defeating dynamic data kernel rootkit attacks via vmm-based guest-transparent monitoring. International Conference on Availability, Reliability and Security, 0:74--81, 2009.Google Scholar
- J. Rutkowska. Linux kernel backdoors and their detection. In ITUnderground, Oct 2004.Google Scholar
- A. Seshadri, M. Luk, N. Qu, and A. Perrig. Secvisor: a tiny hypervisor to provide lifetime kernel code integrity for commodity oses. In Proceedings of twenty-first ACM SIGOPS Symposium on Operating Systems Principles, SOSP '07, pages 335--350, New York, NY, USA, 2007. ACM. Google ScholarDigital Library
- H. Shacham. The geometry of innocent flesh on the bone: return-into-libc without function calls (on the x86). In Proceedings of the 14th ACM Conference on Computer and Communications Security, CCS '07, pages 552--561, New York, NY, USA, 2007. ACM. Google ScholarDigital Library
- A. Shevchenko. Rootkit evolution. http://www.securelist.com/en/analysis?pubid=204792016.Google Scholar
- Solar Designer. Getting around non-executable stack (and fix). 1997. http://seclists.org/bugtraq/1997/Aug/63.Google Scholar
- Trusted Computing Group. TPM Main Specification, 2007. http://www.trustedcomputinggroup.org/resources/tpm_main_specification.Google Scholar
- Z. Wang, X. Jiang, W. Cui, and P. Ning. Countering kernel rootkits with lightweight hook protection. In Proceedings of the 16th ACM Conference on Computer and Communications Security, CCS '09, pages 545--554, New York, NY, USA, 2009. ACM. Google ScholarDigital Library
- D. Wheeler. SLOCCount. http://www.dwheeler.com/sloccount/, 2001.Google Scholar
Index Terms
- Ensuring operating system kernel integrity with OSck
Recommendations
Ensuring operating system kernel integrity with OSck
ASPLOS '11Kernel rootkits that modify operating system state to avoid detection are a dangerous threat to system security. This paper presents OSck, a system that discovers kernel rootkits by detecting malicious modifications to operating system data. OSck ...
Ensuring operating system kernel integrity with OSck
ASPLOS '11Kernel rootkits that modify operating system state to avoid detection are a dangerous threat to system security. This paper presents OSck, a system that discovers kernel rootkits by detecting malicious modifications to operating system data. OSck ...
A normality based method for detecting kernel rootkits
Rootkits are stealthy, malicious software that allow an attacker to gain and maintain control of a system, attack other systems, destroy evidence, and decrease the chance of detection. Existing detection methods typically rely on a priori knowledge and ...
Comments