UNIX System Security: A Guide for Users and System Administrators

From the Book: The UNIX operating system, although now in widespread use in environments concerned about security, was not really designed with security in mind (Ritchie, 1975). This does not mean that UNIX does not provide any security mechanisms; indeed, several very good ones are available. However, most “out of the box” installation procedures from companies such as Sun Microsystems, Digital Equipment Corporation, and AT&T still set up the system in much the same way as it was originally shipped 15 years ago: with little or no security enabled. The reasons for this state of affairs are largely historical. UNIX was originally designed by programmers for use by other programmers. The environment in which it was used was one of open cooperation, not one of privacy. Programmers typically collaborated with each other on projects, and hence preferred to be able to share their files with each other without having to jump over security hurdles. Because the first sites outside of Bell Laboratories to install UNIX were university research laboratories, where a similar environment existed, no real need for greater security was seen until some time later. In the early 1980s, many universities began to move their UNIX systems out of the laboratories and into the computer centers, allowing (or forcing) the user population as a whole to use this new and wonderful system. Many businesses and government sites began to install UNIX systems as well, particularly as desktop workstations became more powerful and affordable. Thus, the UNIX operating system is no longer being used only in environments where open collaboration is the primary goal. Universities require their students to use the system forclass assignments, yet they do not want the students to be able to copy from each other. Businesses use their UNIX systems for confidential tasks such as bookkeeping and payroll. And the government uses UNIX for numerous unclassified yet sensitive purposes. To complicate matters, new features have been added to UNIX over the years, making security even more difficult to control. Perhaps the most problematic features are those relating to networking: remote login, remote command execution, file transfer, network file systems, and electronic mail. All of these features have increased the utility and usability of UNIX by untold amounts. However, these same features, along with the widespread connection of UNIX systems to the Internet (Quarterman and Hoskins, 1986) and other networks, have opened up many new areas of vulnerability to unauthorized abuse of the system. Purpose of the Book Over the years, a great deal has been written about UNIX system security. Papers are presented at almost every USENIX and Uniforum conference as well as others, and the USENIX Association sponsors an annual workshop on UNIX security. Electronic mailing lists exist for the purposes of reporting security problems and receiving fixes for them. Freely available software exists to analyze UNIX systems for security problems and report or correct them. But, in spite of all the work being done in this area, the results of this work have never been collected in one place, and average users or system administrators of UNIX system still do not have the information necessary to protect their data and their systems from unauthorized use. This book provides the information necessary to protect a UNIX system from unauthorized access. The methods an attacker can use to gain access to a UNIX system are described, as well as how to defend against them. In describing the bugs and loopholes used by attackers, it is sometimes necessary to provide information on how they are used. However, we try to strike a midway point between providing enough information so that the reader can understand the problem, while not providing enough information for an attacker to use the text as a cookbook for breaking into systems. The primary focus of the book is on the most recent versions of UNIX (4.2BSD and 4.3BSD from Berkeley) and their most recent vendor derivatives (SunOS4.x, ULTRIX 4.x, etc.), and System V Release 4 from AT&T . However, most of the information presented can be applied, with minor changes, to any version of UNIX since Seventh Edition. An attempt has been made to make the shell scripts and C programs used in the examples as portable as possible by avoiding many of the “convenience” features added in some versions of the operating system. While this means that some of the examples are not as efficient or “clean” as they could be, it also means that they will work on more systems. Industrious readers may feel free to recode any of the examples to use whatever new features are desired. By following the procedures described in the text, and making use of the programs and shell scripts provided as examples, it is possible to secure a UNIX system against most attackers, even if it is connected to wide-area networks such as the Internet.Organization of the Book The book is divided into five parts. In Chapter 1, we begin our discussion by considering four well-known cases of UNIX security being broken. This serves to make the reader aware of some possible types of attack on UNIX systems, as well as to provide a framework for later discussion. Most of the information required to protect a single UNIX system is presented in the next three chapters. Chapter 2 discusses account and password security. File system security is covered in Chapter 3. Chapter 4 describes network security. The third part of the book discusses some topics in more detail. These topics pertain primarily to networks of UNIX systems and systems that provide remote access. Chapter 5 describes the Network Information Service (NIS) (formerly called Sun Yellow Pages, or YP), the Network File System (NFS), and the Remote File Sharing (RFS) service. Workstations and the particular security problems they present are covered in Chapter 6.Terminals, modems, and the UNIX-to-UNIX copy program (UUCP) are discussed in Chapter 7. Chapter 8 describes methods for detecting and responding to break-ins, and serves to separate the discussion of specific UNIX security issues from the material in the last part of the book. The last chapters of the book cover topics that are directly related to UNIX system security, but are not part of the UNIX system itself. Chapter 9 discusses encryption and authentication, and the software and hardware devices used to implement them. Security policies are discussed in Chapter 10. Chapters 11 and 12 discuss publicly available security-related software and how to obtain information about current threats to UNIX security. Background This book assumes that the reader is familiar with the UNIX system, its commands, and its terminology. Readers who are unfamiliar with UNIX should consult A Practical Guide to the UNIX System, Third Edition by Mark G. Sobell, published by Addison-Wesley. Readers who wish to learn more about UNIX system administration (which will be helpful, but not necessary, in understanding some parts of this book) should consult the UNIX System Administration Handbook by Evi Nemeth, Garth Snyder, and Scott Seebass, published by Prentice Hall. Readers who would like to learn more about the history and “culture” of UNIX may wish to consult Life With UNIX by Don Libes and Sandy Ressler, published by Prentice Hall. Acknowledgments First and foremost, I am grateful to my wife, Cathy, and our sons, Trevor and Sean. Their love, patience, and encouragement helped make this book a reality. Many thanks to the people who reviewed the book and provided technical and editorial critiques: Ralph Droms (Bucknell University), Tom Duff (AT&T Bell Laboratories), Fred Grampp (AT&T Bell Laboratories), Dan Farmer (Sun Microsystems), Brian Kernighan (AT&T Bell Laboratories), Rich Kulawiec (Hospital of the University of Pennsylvania), Doug McIlroy (AT&T Bell Laboratories), Michael Merritt (AT&T Bell Laboratories), Jim Reeds (AT&T Bell Laboratories), Jeff Schwab (Purdue University), and Steve Simmons (Industrial Technology Institute). Thanks also to John Wait and Alan Apt, my editors at Addison-Wesley, for their support and assistance while writing the book. A good portion of the information about UNIX security presented in this book has come from discussions with friends and colleagues, and from suggestions received from the readers of an earlier paper I wrote on UNIX security (Curry, 1990). Thanks to all of you, especially Matt Bishop, Dan Farmer, Phil Klimbal, Doug Moran, Donn Parker, Peter Neumann, and Jeff Schwab. Finally, special thanks go to Rich Lynn of Digital Equipment Corporation, who obtained a copy of the ULTRIX Security Guide for Administrators for me, to Ken van Wyk of the CERT/CC, who answered my several requests for information, and to John Wack of NIST, who provided a wealth of information about how to form a response team, and about the FIRST. This book was produced using AT&T’s Documenter’s Workbench, Release 2.0 (eqn, tbl, troff) with the -ms macro package on a Sun Microsystems SPARC station 1 running SunOS 4.1.1. PostScript output was produced by the TransScript package from Adobe Systems. The index was generated using a set of awk and sh programs described in (Bentley and Kernighan, 1988).