Why data-driven decision making is harder than it looks
GettyImages/Shannon Fagan
Connecting state and local government leaders
Producing digestible data to drive more effective decision-making means getting both technology and policies on the same page, a new study shows.
Quality data and advanced analytics software cannot guarantee that governments make effective data-driven decisions, new research finds.
In “Towards Data-Driven Decision-Making in Government: Identifying Opportunities and Challenges for Data Use and Analytics,” researchers from the Center for Technology in Government at the University at Albany (CTG UAlbany) found that agencies also need organizational capabilities to ensure they make the best use of data for and evidence-based policy.
Even though many agencies have embraced data-driven decision making in recent years, not enough studies demonstrate the challenges associated with implementation, UAlbany researcher Yongjin Choi said.
Choi and his team explored a state Division of Water's data usage and analytic practices. During a prototype project, DOW adopted advanced information technologies to collect and analyze data on environmental challenges from harmful algal blooms.
Partnering with the state’s Department of Health and CTG UAlbany, DOW attempted to develop efficient data management practices, suggest governance models and identify better analytical practices as it collected water chemistry and water permit data and developed several databases to manage the data.
The CTG UAlbany team found that while DOW was able to leverage its data for broad planning and designing environmental interventions, it was only used as another source of information for routine decisions.
One challenge was that incompatible and non-interoperable data collection systems and different versions of software and data formatting variations made it unusually time consuming to produce evidence. Plus, the complex questions the water sampling was supposed to answer were beyond the capabilities of the department's legacy technology. The agency struggled to move off its old platform because it needed to analyze long-term trends. Public-sector procurement practices also hindered the agency's ability to adopt modern tools and technologies.
Finally, when the analysis presented to the nonscientific community, the findings did not have the intended impact because they contained too much scientific complexity.
The researchers found nine determinants that could help or harm an agency’s ability to improve its data-driven decision-making:
- Data quality and coverage, compatibility and interoperability.
- IT systems and software and analytical techniques.
- Organizational cooperation and culture.
- Institutional policies for privacy and procurement.
The report suggested that agencies can remedy data analysis problems by having organizational buy-in at each level. This would mean a department leverages all available resources to not only improve analytical capabilities, but also to expand the scope of evidence collection for decision-making.
"Overall, the findings imply that either quality data or advanced analytic techniques alone do not guarantee effective [data-driven decision-making]," the researchers wrote. "Organizational and institutional support is also needed for successful implementation."