The Basic Questions Bedeviling How States Manage Data
Jasmin Merdan via Getty Images
States have tons of data. But much of it is redundant and it’s sometimes hard to know where it is stored, who can access it and how it can be used.
Pennsylvania’s outgoing chief information officer and deputy secretary for information technology, John MacMillan, has what he describes as “skullcracker” questions about state government data.
For example: How much data does each state have? Where is it stored? How is it classified under standards meant to protect residents’ privacy and confidentiality? Who has access to all of this information? And how is it being used?
MacMillan posed these and other queries to a room full of state tech officials—and private sector vendors interested in doing business with them—who gathered recently in Kentucky for a National Association of State Chief Information Officers conference.
“The reason I'm asking,” MacMillan said, “is it's really important.”
“What are you managing?” he added. “Where is it? Then, who has access to it? And should they have access to it?”
MacMillan said that his state has 10 exabytes of data and noted how Pennsylvania, about three years ago, conducted a “manual inventory” looking at every dataset and data element held by the state, across 33 agencies and 2,200 applications. It took around six months to complete.
The information was scattered in different data centers in a number of states. “It changed the day after we finished,” he said. But since then, Pennsylvania has adopted a system that will allow for the inventory of the state’s data to be updated in real time.
MacMillan points out how big tech companies, like Amazon, gain consent from consumers to collect their data one time, but then use it for multiple purposes to cut down on redundancy in how many times information is gathered and stored.
In contrast, governments often collect the same common data elements (like name, address, phone number, email) over and over. This information has to be repeatedly entered by residents and filed away in databases that must then be maintained and then kept secure.
“There are about 35 data elements that are common across almost every customer facing application at every level of government across the United States,” MacMillan said. “How much storage is consumed by just those basic data elements?”
Meanwhile, the format of this data can vary across agencies and programs. Creating extra work and costs when people are interested in matching it up and consolidating it.
Adding a twist, as Bill Kehoe, Washington state’s CIO pointed out, is that unlike frequent Amazon shoppers, not all state residents have easy access to the internet. This complicates matters if states are looking at streamlining and modernizing their data.
“It's not just one way,” he said, adding some people may go into offices, or call in on the phone when interacting with agencies. “There's a lot to this other than just the digital experience.”
Even so, Kehoe sees room for improvement with how states manage their data. "I think data is an afterthought,” he said. "We store a lot of data and we use, like, 1% of it, right? And yet that data is rich in helping us serve our constituents.”
MacMillan makes a case that a good initial step for governments concerned about these issues is to conduct an inventory like the one Pennsylvania did. From there, he suggests there are new opportunities and efficiencies that can be unlocked.
Surely, efforts like this will have to weigh privacy and security, while also setting clear standards for when residents should be asked to give consent to the government for their data to be shared between agencies or, perhaps, funneled into an analysis to improve programs.
Some residents might not be comfortable with that. But if enough are, there could be perks in terms of states providing more convenient access to services, cutting down on the amount of data they have to manage, or improving programs.
Kehoe also noted that if governments don’t have their data in order, it will make it more difficult for them to adopt emerging technology, like artificial intelligence and machine learning.
For MacMillan, though, the seemingly basic—but in many cases deceptively complicated—questions about state data are key.
“You must know where your data is, who has access to it and who can opt in and out of how government uses it,” MacMillan said. “These are modern data concepts that create management nightmares.”
NEXT STORY: Energy data to power state’s clean-energy economy