Water systems’ cybersecurity dogged by ‘turf war’ between IT, OT

A report last year from the Director of National Intelligence found that of the reported cyberattacks on critical infrastructure, more than a dozen were on water utilities, water systems and wastewater treatment facilities.

The cyber threats look to only be multiplying, too. A separate report from the House Homeland Security Committee highlighted “rising threats” from nation-state actors like China, Iran and Russia, with utilities in Pennsylvania and Texas among those hit in recent years.

It puts the onus on critical infrastructure operators and state governments to invest more in cybersecurity to ward off those threats. But a major issue for water utilities remains a tension between the IT systems that manage utilities’ data, applications and networks, and their operational technology that control the physical world and monitor and manage their industrial activities.

“What are the gaps? A lot of it comes from the problem with this issue of IT vs. OT, and the administrators of OT environments do not like IT managing those environments,” said Jake Margolis, chief information security officer at the Metropolitan Water District of Southern California, during a panel discussion at the Billington State and Local Cybersecurity Summit in Washington, D.C., this week. “There are things that exist in their environment that I cannot see. My security operations center doesn't see it, and we cannot protect what we cannot see.”

Andrew Alipanah, chief information security officer for Orange County, California, said on the same panel that it can create the feeling of a “turf war” between IT and OT.

“I'm not trying to tell you how to run your pumps or how to program your [Programmable Logic Controllers, which are used to automate various processes],” Alipanah said. “We just need to be able to see what you're doing on your network, so that we can monitor that.”

That lack of consistency between IT and OT creates “vulnerabilities of opportunity,” said Vitaliy Panych, California’s chief information security officer. But the state has tried to make the two sides work together more coherently, he said, including by building out a hydroelectric plant with equipment donated by an energy company to run drills, tabletop exercises and training. That plant exists within the California Cybersecurity Integration Center, which responds to cyber events, shares information and trains employees on the threats they face.

Another complicating factor in ensuring critical infrastructure is secure against cyber threats is the complex and unclear jurisdictional boundaries the systems sometimes fall under. In California alone, there are more than 1,200 local water districts, which are known as special districts and are generally locally owned. The state then has oversight of those districts, including through the State Water Project, which stores and delivers water across 705 miles through two-thirds of the length of California, and its regulatory agencies. Federal agencies also have a role in managing and regulating water.

“One of the biggest problems that we see, especially in cybersecurity, is the lines of delineation are not obvious and there's always some level of uncertainty going on over there,” said Alipanah. “So oftentimes we don't know whether the local agencies are responsible for something, or whether it is the state or [the municipal water district]. At times, it becomes a point of contention because those issues are not clear exactly, and that's one of the biggest problems that we have locally.”

Margolis said those realities, which are the same in many other states when it comes to overlapping jurisdiction, means entities must work together on common issues like cybersecurity and ensuring water supplies stay in operation.

“You have to recognize that the problem is bigger than one you can handle by yourself in the first place,” he said. “The state doesn't necessarily parachute in to tell you how to manage the water supply between the supply and demand. However, it is a collaborative effort in terms of what we do, and in terms of the shared responsibility model.”

It likely won’t get any easier for water systems, however, especially given the financial pressures they face. A recent report from McKinsey & Company found that utilities face a $110 billion funding gap, which could grow to $195 billion by 2030 as they manage aging infrastructure, more extreme weather caused by climate change and higher usage, in addition to cyber threats.

Panych said the State and Local Cybersecurity Grant Program, which was created under the 2021 infrastructure law, has been a “dose of positive change” in funding cybersecurity locally. But there are still enormous constraints on funds, and it can be hard to, for example, buy cybersecurity services from capital budgets as they are typically subscription-based.

Margolis said utility and government leaders should present cybersecurity as a “line of business” and an “essential service” to systems and for taxpayers. And don’t suggest to those in charge of the money that cyber defenses are impenetrable, as they will always end up disappointed.

“The advice I would give to somebody is to design your program as a resiliency program, not as a pure prevention program, because I hate to break it to you, you can't really prevent a cyberattack,” he said. “Given enough time and effort and determination, an adversary is going to get into your system if they're determined. It's all about resiliency.”