82

The Pseudo-Internal Intruder: A New Access Oriented Intruder Category

A Thesis Presented to

The Faculty of the School of Engineering and Applied Science

University of Virginia

In Partial Fulfillment

of the Requirements for the Degree

Master of Science (Computer Science)

By

Brownell Kerr Combs

May 1999

Approval Sheet

This thesis is submitted in partial fulfillment of the

requirements for the degree of

Master of Science (Computer Science)

______

Author’s Name

This thesis has been read and approved by the Examining Committee:

______

Thesis Advisor

______

Committee Chairman

______

Accepted for the School of Engineering and Applied Science:

______

Dean, School of Engineering and Applied Science

May, 1999

Abstract

Intruders attack both commercial and federal distributed systems frequently, and often successfully. The problem of intruders has become critical. The most effective defense today is the use of intrusion detection systems, because it is widely considered to be impossible to build complicated distributed systems that completely prevent unauthorized intrusions. Since 1980 the intrusion detection community has divided intruders into two categories based on the intruder’s access to a system. Internal intruders have legitimate access through user accounts; external intruders break into a system without benefit of a user account.

The proliferation of distributed systems with complex networks has necessitated a reexamination of intruder definitions. When the notion of internal and external intruders was defined, systems were largely stand-alone computers – typically contained in a single area sometimes with remote peripherals. Today computers are part of networked, distributed systems that may span multiple buildings sometimes located thousands of miles apart. The network of such a system is a pathway for communication between the computers in the distributed system. The network is also a pathway for intrusion.

We define a new category, the pseudo-internal intruder. This new category encompasses intruders without user accounts who circumvent the perimeter defenses of a modern distributed system and attack the system via its network. In contrast, external intruders attack a system from the outside through a system’s perimeter defenses. Having a pseudo-internal category is useful because it gives the intrusion detection community a framework in which to clearly describe the capabilities of the pseudo-internal intruder, defend against the pseudo-internal intruder, and develop techniques for detecting the pseudo-internal intruder.

Acknowledgments

I would like to thank my advisor, Dr. Anita Jones, not only for her guidance and support during the thesis process, but her help and advice with my career.

I would also like to thank Bob Sielken for thought provoking discussions on the topic of intruders, Andy Lowe for help with the case study network and technical proofreading, Chris Milner for advice on numerous tasks around the department, and Peggy Reed for always being willing to lend a hand. I also appreciate the flexibility of my employers at SAIC, Robert Schlansker, Dave Carothers, and William Baugh, in allowing me to split time between work and school.

Thanks and love to Melissa Meehan and Craig Hille for their support and making my time in Charlottesville much more enjoyable.

Lastly, I send my love and appreciation to my mother, father, and stepfather for all of their support and unconditional love since I departed for boarding school some 9 years ago. I could never have achieved this without you.

Table of Contents

Approval Sheet ii

Abstract iii

Acknowledgments iv

Table of Contents v

Figures vi

Tables vii

Chapter 1: The Pseudo-Internal Intruder 1

1.1 Introduction 1

1.2 Literature Survey 2

1.3 Approaches to Categorizing Intruders 6

1.4 Definitions 10

1.5 A New Access Oriented Intruder Category 18

1.6 The Pseudo-Internal Intruder: A Distinct Category? 21

Chapter 2: Capabilities of the Pseudo-Internal Intruder 24

2.1 Tools and Techniques Used by the Pseudo-Internal Intruder 24

2.2 Dangers of the Pseudo-Internal Intruder 31

Chapter 3: Security Recommendations 39

3.1 Defending Systems Against the Pseudo-Internal Intruder 39

3.2 Defending the Distributed System: Preventing Intruder Access 43

3.3 Defending the Distributed System: Mitigating Intruder Access 46

3.4 Defending the Distributed System: Detecting Intruder Access 49

Chapter 4: Case Study 52

4.1 Introduction 52

4.2 The Target System 53

4.3 The Pseudo-Internal Intruder Attacks 57

4.4 Expected Results 59

4.5 Results of Attacks on Target System – Phase 1 60

4.6 Security Changes Made to Testbed System for Phase 2 65

4.7 Results of Attacks on Target System – Phase 2 68

4.8 Summary 72

Chapter 5: Conclusions and Future Work 75

5.1 Conclusions 75

5.3 Future Work 76

References 77

Figures

Figure 1-1: Physical Configuration of Example Network 12

Figure 1-2: Network Configuration of Same Network from Figure 1-1 17

Figure 1-3: Box Diagram of Intruder Categories 19

Figure 2-1: Ipv4 Packet Header 25

Figure 4-1: Network Configuration of Phase 1 Testbed System 55

Figure 4-2: Network Configuration of Phase 2 Testbed System 67

Tables

Table 4-1: Nodes in Testbed Distributed System 53

The Pseudo-Internal Intruder

82

Chapter 1: The Pseudo-Internal Intruder

1.1 Introduction

Each day intruders attack numerous distributed systems. A 1996 report estimated that Department of Defense systems alone are attacked on average over 680 times per day [GAO96]. The report further estimated that as many as 65 percent of those attacks were successful in gaining access to sensitive information. More than 99 percent of all respondents reported at least one security incident in one recent survey [Pow99], while 78 percent of organizations responding to a 1996 survey reported financial loss from security breaches [DV97]. The Computer Emergency Response Team (CERT) annual reports show a 67 percent increase in security incidents handled annually by CERT from 1994 to 1998 [CER94, CER98].

The problem of successful intrusions is not expected to end any time soon. Most experts believe that it is not practically possible to build a complex distributed system that is completely secure. Even if some new development allowed such a system to be created, the vast installed base of vulnerable systems would guarantee a lengthy transition period during which intrusions would still occur [Sun96]. For these reasons security experts advocate the use of intrusion detection systems.

Developers of intrusion detection systems, and those responsible for network security of distributed systems, face the difficult task of defending against an ever changing set of potential intrusions. Each day new attack tools and techniques are developed and it is increasingly difficult for system administrators and intrusion detection system developers to stay ahead of malicious computer users (witness the recent success of the Melissa e-mail virus [CNN99]). Any framework that helps such administrators and developers to understand and classify potential intruders is useful in the struggle to protect distributed systems.

Chapter 1 of the thesis introduces the topic of intruder categories, describes the evolution of intruder categories, and defines important terms and explains aspects of network security that are required to discuss the pseudo-internal intruder. Chapter 1 concludes with a definition and explanation of the pseudo-internal intruder as a new and distinct access oriented intruder category. Chapter 2 lists the tools and techniques available to pseudo-internal intruders and the threat represented by such intruders. Two example intrusion scenarios illustrate the threat of the pseudo-internal intruder. Chapter 3 describes an overall strategy that can be used to defend distributed systems against pseudo-internal intruders. Chapter 4 contains the results of a case study illustrating the effectiveness of the defensive strategy outlined in chapter 3. Chapter 5 contains the conclusions of the thesis and speculates on interesting future work.

1.2 Literature Survey

In 1980 J.P. Anderson introduced the concept of intrusion detection [And80]. Anderson proposed a “security surveillance system” involving formal examination of a system’s audit logs. In examining the system threats, Anderson also introduced the notion of categorizing intruders based upon their access to a system. He noted that in “considering the threat problem, the principle breakdown of threats is on the basis of whether or not an attacker is normally authorized to use the computer system.” Internal intruders were defined as those with permissions to access the system and external intruders were those without any permissions. Therefore the external intruder category included not only outsiders from other organizations, but anyone with physical proximity to the computer system, but without user access to the computer system.

It is important to note that when Anderson wrote his report there were very few distributed systems. In fact, the report that introduced the concepts of intrusion detection and access oriented intruder categories was an actual study of a customer’s single computer system, “the purpose of which was to improve… computer security…” Therefore, it is easy to see why Anderson chose not to differentiate between outsiders and those with physical access, but without authorized user access to the computer system. Whether through tapping wire communication (outsider) or physical access to a terminal (employee without permissions) the best that either could achieve was a login prompt. Both groups of intruders had to deal with the same technical barrier: the system’s access control security measures.

By the mid 1980s the landscape was, however, changing. Distributed systems were becoming predominant over single computer systems. That change started a debate in the network security field as to what changes should be made to existing security principles to adapt them to distributed systems. In 1985 Anderson claimed that “network security issues can be handled with the same concepts that apply to single computer systems” [And85].

Anderson’s opinion appeared to be in the minority. Nessett argued that “a strong case [could] be made that distributed systems admit important security issues that either are not applicable to stand-alone systems or are assumed to be rarely relevant… [Such] issues add extra dimensions to the distributed system security problem and invalidate attempts to simply extend existing concepts into the area of distributed system security” [Nes87]. Others pointed out that any protection mechanism residing in a single computer becomes insufficient when a computer is connected to a network because those mechanisms cannot protect the security of communication across the network [LS90]. Such distributed systems require a security enforcement mechanism for the network in addition to any mechanisms residing on single machines [LS90].

There was, however, no dispute that both stand alone and distributed systems needed real time intrusion detection. In her paper introducing a new intrusion detection model, Denning argued that “developing systems that are absolutely secure is extremely difficult, if not generally impossible. [Additionally], even the most secure systems are vulnerable to abuse by insiders who misuse their privileges” [Den87]. Denning’s model of intrusion detection was an adaptation of Anderson’s original idea of utilizing audit logs for intrusion detection. Since “exploitation of a system’s vulnerabilities involves abnormal use of the system,” intrusions can be detected by monitoring audit logs and other indicators for abnormal patterns of system usage [Den87].

Denning’s model of intrusion detection is considered to be the beginning of the second generation of intrusion detection which was more statistically sophisticated, addressed distributed systems, and provided some real time alerts [JS99]. This second generation of intrusion detection systems is divided into two approaches: anomaly detection and misuse detection. Intrusion detection systems based on anomaly detection characterize the correct behavior of a system and then detect wrongful changes to that correct behavior. Misuse detection systems characterize known ways to penetrate a system and then monitor for those misuse characterizations to appear. This progression of intrusion detection can be more closely followed in a number of recent surveys of intrusion detection techniques and products [CH96, Lun93, Sun96, JS99].

A majority of intrusion detection experts currently believe that the best intrusion detection system will contain both anomaly detection and misuse detection mechanisms. One such intrusion detection system that includes both anomaly detection and misuse detection mechanisms is the Next-generation Intrusion Detection Expert System (NIDES) [AFV95]. NIDES contains a statistically dynamic anomaly detector to catch internal intruders masquerading as legitimate users. A profile consisting of more than 30 different criteria (such as CPU usage and typical amounts of input and output) is maintained for each user. User actions are matched against that individual’s profile and “when the observed activity departs from established patterns of use for an individual” alarms are raised by the intrusion detection system [SRI97]. The mechanism is statistically dynamic (as opposed to static) since NIDES adapts each user’s profile over time. If a user’s habits change slowly over time, the profile will be adapted to the new behavior without raising alarms. Profiles can also be created for workstations, remote hosts, groups of users, or particular application programs [JS99].

NIDES also contains an expert system misuse detection to “detect attempts to exploit known security vulnerabilities of the monitored systems and intruders who exhibit specific patterns of behavior that are known to be suspicious or in violation of site security policy” [SRI97]. NIDES observes the system and compares its observations to a rule database of known intrusion scenarios and attack patterns. The security experts that created NIDES initially constructed the rule database, but the system administrator of the system NIDES is protecting can customize the database.

Even though it is accepted that distributed systems require different security mechanisms than stand-alone systems, the intruder categories defined for stand-alone systems are still in use. Many modern intrusion detection system research papers still describe the threat of intruders as non-authorized (external) and authorized (internal) users [IKP95]. Recall that in Anderson’s seminal paper he only evaluated the threat of an intruder with respect to whether the intruder had authorized user access to the computer. As discussed, this made perfect sense when considering a stand alone computer. But with the proliferation of the distributed system, this way of categorizing intruders should be reexamined for distributed systems in a manner similar to the reexamination of security concepts.

1.3 Approaches to Categorizing Intruders

There are two main approaches to classifying intruders. The first is to simply separate intruders into categories based on their access to a system. An example of this approach is the previously discussed traditional pair of categories: external and internal intruders [And80]. The external intruder is an outsider who has no authorized access to the system and must gain access by compromising the system’s security. The internal intruder is one who already has limited access to the system through an authorized user account. An internal intruder can either be a legitimate user or an outsider who is successfully masquerading as a legitimate user.