How we tested: GCN Lab partners with network analysts
Connecting state and local government leaders
The GCN Lab partnered with Spirent Communications of Calabasas, Calif. to help test antispam and anti-virus network appliances.
The GCN Lab partnered with Spirent Communications of Calabasas, Calif. to help test antispam and anti-virus network appliances. Spirent () makes a line of equipment for testing network devices and systems.For our tests, Spirent provided two pieces of equipment: The Avalanche, which is able to generate and send large volumes of e-mail traffic, and the Reflector, which simulates any size network. When testing network appliances, we place the product between the Avalanche and the Reflector and measure its performance.For this review, we built a simulated government network of 1,000 users. We subjected each network appliance to three different performance tests. For each one, we conducted several tuning tests to normalize the appliance. If accuracy rates were within 0.5 percent on two separate tuning tests, we then conducted one final test that produced the product's actual performance score. The final test also had to fall within the 0.5 percent margin.For the first test we sent a stream of spam e-mail through each appliance. We sent the spam from a pool that was not more than two days old and came directly from several drop boxes on the GCN Lab network. The test simulated a real-world environment and lasted for 10 minutes. As such, the number of e-mails going through each device during the test varied a bit. What we looked for was the percentage of spam messages each appliance successfully quarantined.The second test mirrored the first, but used a pool of several thousand viruses, which were delivered as attachments to normal mail.The final test was a mixed stream of good mail that we created, spam and viruses. We checked spam queues to see if any of the good mail was falsely quarantined.In addition to performance testing, we looked at the user interfaces of each appliance's various software tools. Network appliances at this level are exceedingly complex, because they can be set to guard against a lot of threats and adjusted to scan mail in many different ways. Having a clean, easy-to-use interface is the key to getting the most out of any appliance.It's worth noting that although four companies agreed to be in this review, many more were invited. Some companies did not have review units available that matched our 1,000-user test-bed requirements.Some said they did not want to be tested head-to-head with their competitors. Most companies told us they'd never been invited to a review of both their antispam and antivirus capabilities, and as such were wary of participating.That said, we're currently gathering additional network appliances for testing with our Spirent equipment. Those reviews should appear in future issues of GCN.For a complete list of other tests Spirent has conducted or helped conduct, visit the company's site, .
NEXT STORY: Security appliances keep mail stream clean