Can automated security put agencies a step ahead of the hackers?

Nevada’s Transportation Department, like many other agencies, is better at generating logs and other security information than using the data. Reviewing log data ideally is a routine process for administrators and security professionals, but in the real world the volume and complexity of the data can make it hard to keep up.

“We do not have the resources to do it on a regular basis,” said Kimberly Munoz, the Nevada DOT’s IT manager.

Munoz wanted something that could bring the data together into a central, easily searched location, “like a Google of our system data,” she said. The technology existed, but with budgets tight, getting the funding for it was a challenge. Fortunately, “I was able to lobby and get this on the list,” said Munoz, who sits on a statewide security committee.

In the fall of 2012 the department began a demonstration with Splunk Enterprise, a tool to gather and correlate structured and unstructured network data and make it visible and available for search and analysis. Initial plans were to enable better reporting of the department’s content filtering and to document Web activity.
 
But even in the trial stage, Splunk began finding misconfigured network devices almost immediately. On a firewall that had just been deployed that should not have allowed outside Secure Shell connections, Splunk alerted the team to a number of failed connection attempts from China. The configuration was fixed within a matter of hours.

“It definitely has improved our security posture,” said Munoz.

Splunk is an example of a tool that can automate, or help to automate, routine and cumbersome tasks that are difficult or impractical to do manually. It integrates various types of data in different formats and makes them visible through a “single pane of glass.”

“It is used in IT operations to help with network and systems management, to monitor and understand the use of applications and for operational intelligence,” said Bill Cull, Splunk vice president of public sector. But, “the primary thrust of what we’re doing has been in cybersecurity,” he said.

It’s also helped DOT troubleshoot some problems. When a remote office was having difficulty connecting to the network, “we had a really hard time detecting what the problem was,” Munoz said. “We had spent six weeks and were no closer to a resolution. Splunk fixed it in a week.”

The department has 2,000 employees and oversees 5,400 miles of highway and more than 1,000 bridges, and also operates the state’s public 511 road service and a statewide video traffic network. These systems have generated as many as 35,000 errors an hour in the past, but the visibility provided by Splunk has helped reduce that to 2,500 errors a day.

The DOT currently has a license to allow indexing of one gigabit of data a day, which is not enough for all the purposes Munoz would like to use Splunk for. “We’re only scratching the surface,” she said. “But it’s very useful to us.”

The demand for automation in cybersecurity is not surprising. Attackers increasingly are using automated tool kits to launch attacks, searching for vulnerabilities and weak passwords and then deploying multistage exploits to breach systems and escalate their privileges on it. Although many enterprises detect and block thousands of probes and attempted breaches daily, many attempts make it through perimeter defenses, and once inside a system they can hide themselves well.

Steve Hanna, distinguished engineer at Juniper Networks, said that 60 percent of successful breaches are completed within a day, and 46 percent of them within an hour. Discovery, on the other hand, often is a lengthy process. Fifty-six percent of these breaches are not discovered for more than a month after they occur, and many of them are discovered not by the system owners but by external parties — the latest figure from Verizon’s 2013 Data Breach Investigation Report is 69 percent.

While attackers are commercializing and automating their attacks, government IT shops are struggling with static or shrinking budgets and a shortage of trained, experienced security professionals. Security automation is emerging as a means of catching up with, and maybe getting ahead of, the bad guys.

Security is not a point process, however, and security automation is not accomplished with a single product. It is a broad approach that uses standards-based tools to handle routine tasks automatically, taking humans out of processes that can be done automatically and freeing them for the jobs that require a human touch, such as in-depth analysis or response decisions.

Hanna recently described a four-step model for an effective security program:

1. Prepare, which includes risk analysis and development of policies, controls and training.
2. Detect, which includes monitoring configuration, detecting intrusion and anomalies and sharing threat information.
3. Analyze, which includes incident management, correlation and human analysis.
4. Respond, which includes containment, evidence gathering, recovery, forensics and prevention.

The entire cycle can be automated, or individual elements of this process can be automated separately. Essential to making automation work when a variety of tools from different vendors are being implemented is the use of standards to help ensure interoperability and consistency.

NEXT: How NIST, DHS are moving the ball forward.