Controlled Chaos

The Homeland Security Department isn't just about protecting borders and preventing another 9/11. As the research and development arm for DHS, the Science and Technology Directorate has the mission of developing a technological edge that would also help fight terrorism. The office has designated cybersecurity as an area worthy of attention. Not only could the government benefit from greater network security tools, but DHS also expects its work could help protect commercial networks and critical infrastructure.When it comes to network security, DHS has found that studying must come before defending. The agency's Science and Technology Directorate has co-funded, along with the National Science Foundation, two test beds that researchers may use to replicate network attacks. Such test beds may provide the essential tools for fighting tomorrow's computer attacks.The test beds are ideal for running 'risky' code, or viral programs that could propagate out of control and infest all the nodes of a given network. Vendors can test their new products on these networks, and researchers can test their new experimental code.Both test beds are funded jointly by DHS and NSF, under a program called the Cyber Defense Technology Experimental Research project, or DETER. The University of Southern California, University of California at Berkeley and McAfee Inc. of Santa Clara, Calif., manage the test beds.'Our goal is to have a test bed that can run truly live malicious code,' said Terry Benzel, deputy director of the Computer Networks Division at USC's Information Sciences Institute.The test beds were only a first step. The directorate has also funded work to make future network simulations more exact. A related program is gathering real-life samples of data traffic that can be used on the test beds. And a third program is developing a set of metrics that can be used to scientifically determine the effectiveness of various experimental approaches.'The idea is to provide large-scale test beds that emulate a real architecture,' said Annabelle Lee, Science and Technology Directorate portfolio manager for cybersecurity.Lee considers DETER one of the early success stories of DHS' cybersecurity research funding. The cybersecurity portfolio has a modest budget: $18 million for fiscal 2005 and a proposed 2006 budget of $16.7 million. Lee's office takes lists of prioritized requirements from other DHS directorates, most notably the National Cyber Security Division and the National Communication System, and forges them into areas of interest for which DHS will fund new research.Live since March 2004, the DETER test beds have already hosted a number of experiments. There may be 15 to 20 experiments running at one time, Benzel said. One project simulated how the Slammer worm affects network behavior. Another tested a software prototype that could detect and redirect distributed denial-of-service attacks.In many ways, it was this experiment that validated the need for a large-scale test bed, Benzel said. Historically, researchers have set up smaller test beds, with maybe a dozen nodes, to work out network security problems. Distributed denial-of-service attacks, however, involve hundreds or thousands of computers.Each test bed is contained in a single room. Researchers can either visit the facilities and conduct experiments, or run them remotely over the Internet. There is a 72-node cluster at USC, while another 32-node cluster operates at Berkeley.DETER uses software called Emulab, a network simulation platform developed by the University of Utah. The networks themselves employ a combination of computers, including IBM Netfinity servers and Sun Microsystems Sun Fire machines. Each rack-mounted computer has five Ethernet cards. Four are used to represent separate nodes and a fifth is set up as an out-of-band port to control the machine. The nodes themselves connect to different Ethernet switches depending on location (a Cisco 6509 switch at USC and a Foundry Fast Iron 1500 switch at Berkeley).At the beginning of each experiment, the user picks from a list of scripts that model different network environments. The Emulab software loads the operating system onto the nodes and carves out virtual local area networks for testing. The computers themselves can be set up as routers, to generate traffic, or as end-user systems.One of the most difficult challenges was configuring the test beds to make them accessible over the Internet, Benzel said. Allowing researchers to run experiments remotely was a desirable feature, yet the design team had to ensure that any potentially damaging code run on the test beds could not jump over onto the Internet.According to George Kesidis, one of the principal investigators on the project, 'It is a doubled-edged sword. You want to make it quarantined from the Internet, but you don't want everyone to fly to [USC] in order to use it.' So the design team included an intermediary machine that can be tapped over the Internet, in which the experiments can use a Secure Shell session to connect to experimental nodes.DETER has been just one part of DHS' efforts to simulate realistic network traffic and the attacks that plague it. The program has successfully established the hardware and supporting software required for large-scale experimentation. But its development immediately sparked additional avenues of R&D. Researchers needed to know what standard metrics are required to gauge the effectiveness of possible mitigation efforts. And how do you simulate normal day-to-day traffic on an experimental network?DHS found that neither concern was addressed adequately in the commercial sector. So the agency, again in conjunction with NSF, funded two additional projects to tackle these issues.To address the first set of concerns, the Evaluation Methods for Internet Security Technology (EMIST) program is developing a set of scientifically rigorous testing frameworks. It will look for ways to run test bed experiments so that they will be scientifically repeatable and correlate with actual, real-world conditions.In order to develop such metrics, the program itself has been a pioneering user of DETER. This project is carried out by the coalition of academic institutions and commercial companies.'We're discovering problems with the test bed so when we open it up to a much wider community of experimenters, they won't discover these problems,' said Kesidis, who, in addition to his work on DETER, is one of the principal investigators on EMIST.One area that could use some standardization, for instance, is the amount and type of background traffic experienced every day by a network.'Previously, each researcher defined their own way of seeding the experiment. Now EMIST provides a common interface to a common set of tools so we can all see common types of experiments in the same way,' Benzel said.A third program, Protected Repository for Defense of Infrastructure against Cyber Threats, or PREDICT, also addresses this issue.This effort is collecting 'a set of data sets and actual network data that can be used by DETER to test tools and software,' Lee said. The data sets are being obtained from Internet service providers and telecommunications companies. These data sets can be used to model, or even replicate, data packet traffic on the test beds, which then can be subjected to attacks and prototype defense measures.DHS is hoping that, taken together, DETER, EMIST and PREDICT will provide a solid base for better understanding network security. Even building the test beds, testing tools and simulation data sets can lead researchers to think about the nature of future threats. After all, they must build tools that can be useful in the years to come.'How do you rigorously test defenses without coming up with new attacks?' Kesidis said. 'Along the way, we're debating what the true threats of the future are.'