What We Know About the SolarWinds Breach

Federal agencies are responding to a potentially massive cybersecurity incident after a third-party vendor widely used across government was found to be compromised by a sophisticated hacking campaign.

With agencies working to determine which systems were affected and how severely, a myriad of questions swirled. Here’s a look at what we know and what’s been reported as of Tuesday afternoon.

National Security Council Invokes PPD-41

On Tuesday, the National Security Council announced the administration would be invoking Presidential Policy Directive-41, or PPD-41, which “facilitates continuous and comprehensive coordination for whole-of-government efforts to identify, mitigate, remediate and respond to this incident,” according to a statement from NSC spokesperson John Ullyot.

The directive, signed in the last years of then-President Barack Obama’s administration, creates a chain of command for responding to cyber incidents. The directive states the White House Cybersecurity Coordinator—or “an equivalent successor”—will serve as the chair for a Cyber Response Group to develop a strategy while the Cyber Unified Coordination Group will coordinate between federal agencies. 

Within the CUCG, the directive designates the FBI and the National Cyber Investigative Joint Task Force, the Department of Homeland Security’s National Cybersecurity and Communications Integration Center, and the Office of the Director of National Intelligence’s Cyber Threat Intelligence Integration Center as the lead agencies for certain types of responses. Agencies, however, maintain operational control over their networks, “unless mutually agreed upon by agency heads or their designees.”

Since the policy was signed, a few organizational changes have occurred. In 2018, then-National Security Adviser John Bolton eliminated the White House Cybersecurity Coordinator position and the Cybersecurity and Infrastructure Security Agency—previously known as National Protection and Programs Directorate—became a standalone agency.  

The Office of the Director of National Intelligence confirmed they would be joining the Cyber Unified Coordination Group, as outlined in PPD-41.

“The Intelligence Community is marshalling all of its relevant resources to support this effort and share information across the United States Government,” a spokesperson told Nextgov.

Agencies Ordered to Disconnect SolarWinds Products 

Federal agencies were in a rush Monday to turn off and disconnect any and all systems sending data to the SolarWinds Orion product, which helps IT managers monitor and manage their networks. Per an emergency directive from CISA, agencies had until noon to turn off all instances of Orion, search for any evidence of compromise in the connected systems and report in.

The full scope of the intrusion and what it means for the federal government remains unclear. However, the fact that a trusted service provider used by the majority of federal agencies was sending vulnerable code that could further compromise those agencies is likely to reverberate through the federal community for some time.

The incident puts the supply chain problem in full view, along with reigniting arguments about building software in-house over buying it off-the-shelf that had gone dormant in some parts of government.

Details About the Incident

As a publicly-traded company, SolarWinds submitted a report to the Securities and Exchange Commission detailing what the company knew as of that date. At that time—and as of publishing—the vulnerability appeared to be in code updates pushed from March to June of this year.

Per the filing, SolarWinds “currently believes” that users who updated the software between March and June would have downloaded the vulnerability. However, the problem “was introduced as a result of a compromise of the Orion software build system and was not present in the source code repository of the Orion products,” the company states.

The investigation is ongoing, but this could mean users who did not download the updates between March and June were not affected.

According to the filing, SolarWinds claims to have more than 300,000 customers. Of those, 33,000 “customers that were active maintenance customers during and after the relevant period”—in other words, customers that would have received a push to update their code—were notified of the vulnerability on Dec. 13.

However, a little more than half of those customers do not appear to have updated their systems during that time.

“SolarWinds currently believes the actual number of customers that may have had an installation of the Orion products that contained this vulnerability to be fewer than 18,000,” the filing states.

The SEC filing also notes the company is working with the FBI, intelligence community and “other government agencies in investigations related to this incident.”

The Agencies Affected So Far

While this is very likely to affect many federal agencies, only one has confirmed an incident related to the vulnerability: the Commerce Department’s National Telecommunications and Information Administration, or NTIA.

In an interview with Fox Business, later tweeted by the Commerce Department’s official handle, Commerce Secretary Wilbur Ross confirmed a breach had occurred and said more information would be coming.

“We really can’t get into further details because if we did it would give them a clue as to how far along we are with understanding how they did it,” he said. “There will be more revelations soon. But for the moment we’re taking it very, very seriously.”

Other federal agencies issued statements—publicly or directly to Nextgov—acknowledging the severity of the issue but falling short of confirming a breach at their agencies.

“The DOD is aware of the reports and is currently assessing the impact,” a spokesperson for the Defense Information Systems Agency told Nextgov. “The NSA and JFHQ DoDIN,” the DOD’s internal network, “has issued guidance and directives to protect DOD networks and IT systems.

“For operational security reasons the DoD will not comment on specific mitigation measures or specify systems that may have been impacted,” the spokesperson added. “The DoD has some of the best trained and resourced cyber and IT professionals in the world, and we are proud of their tireless efforts to protect our networks.”

The NSA offered a similar statement to Nextgov: "The National Security Agency is working closely with our U.S. government partners regarding the widespread cyber incident affecting a variety of entities and systems. As part of a coordinated, whole of government response effort, NSA is providing technical support and expertise to our partners. NSA has provided mitigation guidance to our Department of Defense, Defense Industrial Base, and National Security System cybersecurity customers. We encourage all cybersecurity stakeholders to take immediate action to protect their systems."

Homeland Security Assistant Secretary for Public Affairs Alexei Woltornist acknowledged the existence of breaches—not just vulnerabilities—Monday afternoon, though did not confirm which agencies were affected.

“The Department of Homeland Security is aware of cyber breaches across the federal government and working closely with our partners in the public and private sector on the federal response,” he said in a statement. “As the federal lead for cyber breaches of civilian federal agencies, the Department of Homeland Security’s Cybersecurity and Infrastructure Security Agency has already issued Emergency Directive 21-01 to the federal government to address compromises related to SolarWinds. As further remedies to these vulnerabilities are available, CISA will update the public at www.cisa.gov.”

Woltornist’s statement also included a request for anyone with additional information to contact CISA at central@cisa.gov.

The Veterans Affairs Department told Nextgov the agency is actively investigating the incident.

“VA is looking into this issue and currently there are no signs of exploitation,” a spokesperson said Tuesday. “However, we have taken SolarWinds offline out of an abundance of caution.”

While Commerce has confirmed its breach, reliable news outlets have been calling out additional agencies, including DHS, the Defense Department, National Institutes of Health and the Treasury Department. None of these agencies have confirmed breaches publicly yet.

Who Is Behind It?

Reports from Reuters—which first broke the story—and the Washington Post point to Russia as the culprit, specifically a government unit known globally as APT29, or Cozy Bear. 

Federal authorities have yet to name suspects publicly.

In its report on the intrusion, cybersecurity firm FireEye—which itself was breached in the hacking campaign—called the perpetrators a “highly evasive attacker” but did not name a specific group or country.