The Last Line of Defense: A Comparison of Windows and Linux Authentication and Authorization Features

The Last Line Of Defense: The Operating System

The number of computer security incidents reported from various forms of attacks has increased significantly since the introduction of the Internet (CERT1; Yegneswaran, Barford, & Ullrich, 2003). Though it is clear that the introduction of the Internet coupled with the decreased cost of networking has helped to pave the way for attackers, the end result of most malicious attacks is the alteration of the host operating system. This alteration is often with the intent of propagating the malicious program and continuing the attack (virus, Trojan horse) or potentially damaging, stealing or altering some content on the host machine. While this type of attack may be aided by the network and security weaknesses therein, the attack could not be successful without ultimately compromising the host operating system. While much attention has focused on securing the network, since it is ultimately the operating system which is compromised, a closer examination of the defense mechanisms of the operating system may be warranted (Losocco, Smalley, Mucklebauer, Taylor, Turner, & Farrell, 1998). Security weaknesses in host operating systems are therefore a major concern for the IT practitioner. If unwanted modification of the host system can be prevented, then the attack may be thwarted despite any weaknesses in the network which allows the attacker to contact the host machine.

There has been a distinction drawn in research between application security and operating system security. It has become increasingly clear, however, that such a distinction is academic and that in practice malicious programs and the individuals who create them make no such distinction. Malware such as Code Red exploited weaknesses in both application and operating system security (Staniford, Paxson, & Weaver, 2002). What is required is an end-to-end solution, one that considers not only the distributed nature of the current computing environment and the network, but the close relationship between the application program and the operating system (Howell & Kotz, 2000; Saltzer, Reed, & Clark, 1981; Thompson, 1984). Recent focus on the concept of endpoint security represents an additional effort to understand and mediate these risks (Kadrich, 2007).

This chapter will examine specific security features of the host operating system in a descriptive and exploratory manner. By understanding the security controls available at the operating system level and the security weaknesses in those systems it is possible understand how to better prevent attacks on these systems.

Operating systems and their underlying security mechanisms are clearly a varied landscape which over time can be quite fluid. This chapter will focus on two common server operating systems: Microsoft Windows Server 2003 and Red Hat Enterprise Linux Server 5. Rather than refer to specific versions of these operating systems, this chapter will use the terms Windows and Linux to refer to Windows Server 2003 and Red Hat Enterprise Linux Server 5 respectively. (As this chapter goes to press, the next version of Windows server operating system, Windows Server 2008, is in Beta 3; the updates to security features in this release considered relevant to this discussion will be identified and evaluated in this chapter.)