19 July 2000

Defending Yourself: The Martial Arts of Intrusion Detection

The following article is excerpted from volume 1, issue 4 of "Infosec Outlook", a joint monthly publication of the Information Technology Association of America and CERT Coordination Center at the Software Engineering Institute.

To learn how to defend themselves against attackers, many people pursue martial arts training.... Intrusion detection follows similar principles. Although intrusion detection technology is immature and its effectiveness is limited, it is useful as one portion of a layered enterprise-wide security plan.

The Attacks

Intrusion detection is becoming increasingly important as the stakes become higher. In the 1980s and early 1990s, denial-of-service (DoS) attacks were infrequent and not considered serious. Today, successful DoS attacks can shut down e-commerce-based organizations like online stockbrokers and retail sites. By our definition, an attack must include both an overt act by an intruder and a manifestation, which the intended victim can observe, that results from that act.

The Defense

The goal of intrusion detection systems (IDSs) is to characterize information to identify true attacks without creating false alarms. Intrusion detection can be viewed as a traditional signal detection problem. Intrusion manifestations are the signals to be detected, while "normal" manifestations are noise. A decision process must determine if an observation is just noise or if there is a signal amidst the noise. Intrusion detectors typically base their decisions either on signal (signature-based detectors) or noise (anomaly-based detectors) characterizations. Each approach has strengths and weaknesses, but both have difficulty characterizing the distributions.

In order for a signature-based IDS to detect attacks, it must be able to identify specific patterns within a single network packet or over multiple packets. If it locates one of these patterns, a signature- based system can identify unseen attacks that are abstractly equivalent to known patterns. But, since they can only identify patterns they have been programmed to know, signature-based systems cannot detect novel attacks. They also suffer from false alarms when they misinterpret signatures. Signatures can be developed in a variety of ways, from hand translation of attack manifestations to automatic training or learning using synthetic sensor data. Because a given signature is associated with a known attack abstraction, it is relatively easy for a signature-based detector to assign names to attacks. IDS products based on current signature-based analysis produce useful results in specific situations, but since they cannot detect novel attack patterns, they do not provide a complete intrusion detection solution.

Anomaly-based detectors equate anything that does not appear to be just noise with intrusions. The primary strength of anomaly detection is its ability to recognize novel attacks. Its drawbacks include the necessity of training the system to separate noise from natural changes in the distribution. Changes in standard operations may cause false alarms while intrusive activities that appear to be normal may cause missed detections. It is also difficult for these systems to name types of attacks.

Making Choices

Depending on your needs, there are commercial and non-commercial tools; each has advantages and disadvantages. Commercial tools can be purchased from various vendors; non-commercial tools are available free of charge.

Since the new product cycle for commercial IDSs is rapid, information and systems quickly become obsolete. One expert recommends using product guides that are updated at least monthly; you can also check web sites. Public domain systems are unlikely to have the same level of support as commercial systems, so you need a higher level of technical expertise to install and manage them. However, this added effort results in a better understanding of intrusion detection's strengths and limitations.

Staff of the CERT/CC condu