Buyer beware: Avoid immature AI solutions
Laurence Dutton/Getty Images
Connecting state and local government leaders
Given the sensitivity of the data state and local governments handle, procurement officers must educate themselves on the fundamentals of artificial intelligence and use caution when considering AI services.
Training procurement officials on emerging information technologies such as artificial intelligence may be necessary to ensure they get the best options for their agencies, experts say. But there’s no rush.
“What we’re talking about is something that we don’t know that much about yet, and when you consider what the state agencies are tasked with, which is handling sensitive information, health information, school information, personal identifying information, financial information, I think it’s only responsible to be cautious,” said Rebecca Montano-Smith, deputy chief learning officer at the National Association of State Procurement Officers (NASPO), which offers educational opportunities.
In fact, the White House May 4 issued actions to promote responsible AI, including calling on tech companies to make sure their products mitigate risk to people’s rights and safety when they are deployed.
State and local officials need to take time to understand several factors before procuring AI, agreed Kevin Desouza, professor of business, technology and strategy at the Queensland, Australia, University of Technology and nonresident senior fellow at the Brookings Institution.
“Procuring AI systems is not the same as your traditional (vanilla) information systems,” Desouza wrote in an email to GCN. “Procurement officers need to know 1) how AI systems are different (more complex) that traditional systems, 2) understand the risk/opportunity tradeoffs, and 3) understand the long-term costs required to keep updating the systems (so that they continue to learn) and maintaining them (so that they perform at desired levels).”
Training is one way to tackle those requirements. Regardless of what procurement officers are buying, they must educate themselves on the product or service and its market so that they know what suppliers exist, where supply-chain issues are and what risks come with certain contracts or products, said Matthew Oyer, chief learning officer at NASPO.
For AI specifically, NASPO is dipping its toes into training. It added a lesson on the potential uses of ChatGPT to an IT procurement certificate program it recently wrapped up with Arizona State University, and it might add an online course about AI.
Oyer likened NASPO’s approach to AI to its handling of blockchain a few years ago. “It was a similar conversation of, ‘Hey, we don’t really know what this is. We’re hearing it could impact state government, it can impact procurement,’” he said. “Our response was similar to what we are doing with AI,” he said. The association found experts to speak to its members about the basics of the technology, common vocabulary and any examples of blockchain’s use in government procurement.
Planning is another crucial aspect of AI procurement, Desouza added. A 2022 report he coauthored studied federal spending on AI, but a big takeaway from that was the need for a coordinated strategy for AI initiatives.
“The role of government at any level is to facilitate innovation across the entire ecosystem,” he said. “To do this, it is important to set an overarching framework that the various players in the ecosystem can buy into and coordinate. This is missing at all levels in the U.S.”
Currently, agencies’ procurement of AI is mostly for experiments, Desouza added, putting them at Level 2 on the “Maturity Framework for AI” that he developed with the IBM Center for the Business of Government. Level 1 means that agencies are largely unaware of AI, while Level 4 means an agency is actively deploying AI systems to scale. The highest level—5—comes once agencies have a robust system in place.
That experimentation is appropriate, Montano-Smith said. “This is brand-spanking new. It’s not really ready for rollout,” she said. “ChatGPTs … where it’s just making things up wholesale, I mean, that doesn’t inspire confidence. It’s going to be a while, I think, before a discrete product service comes along that can be rolled out at a state level with confidence.”
In time, however, AI can help procurement officers themselves, Desouza noted. “AI can actually help to streamline the contracting process by automating some of the bureaucracy,” he said.
But there are risks, he added: “It comes down to deploying systems that are not mature for use. Data and algorithms are the two key ingredients when it comes to AI systems. If the data is not representative of the problem or the algorithms cannot be contested or inspected, one will run into a whole wrath of issues,” he said. “These issues will be in addition to what one would normally expect when digital transformation projects fail.”
Stephanie Kanowitz is a freelance writer based in northern Virginia.