ABSTRACT

Network security has enjoyed a growth in interest as the number of Internet users, and threats, has increased. However, the expected performance for any given security detector cannot be predicted, nor can the ability of the detector to recognize events with differing characteristics be quantified. Further, comparing two different detectors in order to determine which will perform best under a given set of conditions is not easily performed. Rather, the performance results for security detectors, when presented at all, tend to be based on either results from testing using MIT's Lincoln Labs data set or from tests using network traces captured from a single network block. The first approach, while providing a useful baseline for side-by-side comparisons of security detectors, consists of an aging data set that has well-known shortcomings, while the second approach does not address how well a detector will perform in an environment that differs in size, usage or design.

In this presentation I will start with examining some of the issues involved in testing network security detectors. I will then present a new testing methodology, which I used to test a co-ordinated port scan detector. The end result from this methodology is a regression model that describes how well the detector performs in the test environment given scans with varying characteristics. I demonstrate the model's predictive capability when faced with a new operating environment, as well as illustrate its ability to compare the capabilities of two different detectors.

BIOGRAPHY

Carrie Gates is a Research Staff Member with CA Labs on Long Island, New York. Her role is to perform enterprise security research in collaboration with university faculty and students with the aim of providing research results that have strategic value for CA. Previously, she was a member of the technical staff at CERT, Carnegie Mellon University, where she did security research for large scale networks. Her research interests include the detection of security events using network traffic, network visualization approaches for security administrators, usable security and privacy.