Shining a light on shadow AI

Laurence Dutton/GettyImages

 

Connecting state and local government leaders

Employees sometimes use artificial intelligence websites like ChatGPT to get more work done. But if they do so without approval, they may be putting their agency at risk.

Under pressure to do more with less—and do it faster—government workers may be experimenting with online artificial intelligence solutions, whether their information technology teams know about it or not. The result is shadow AI, and it puts agencies at risk.

Just as shadow IT refers to devices, software and services that employees use without the IT department’s knowledge, shadow AI systems operate within an agency unbeknownst to the people responsible for managing technology use. 

The explosive popularity of the generative AI tool ChatGPT has made minimizing shadow AI even harder, said Amy Glasscock, program director of innovation and emerging issues at the National Association of State Chief Information Officers.

“It used to be that if the state was using some sort of artificial intelligence product or service or tool, they would have had to go to an outside vendor and have a contract and … a set number of licenses,” Glasscock said. “But now anyone can go sign up for ChatGPT or some of these others and it’s just so accessible to everyone. It’s very easy for you not to know what employees are using generative AI.”

A main reason why employees turn to AI is to augment their efficiency. In fact, 41% of employees have acquired, modified or created technology without the oversight of IT departments, according to a Gartner report, which predicts that figure will hit 75% by 2027. 

And the technology seems to deliver: Generative AI alone could have a productivity effect of about $480 billion in the public sector, according to McKinsey & Co.

Some workers adopt AI on their own because they feel that their agencies aren’t moving quickly enough to approve AI technologies, but others may not know that they need to go through IT, Glasscock said. “With generative AI, I think employees might just be thinking, ‘Well, this is just a website out there,’” she said.

Unsanctioned AI use runs the risk of exposing data collected by and entrusted to government. If employees feed personally identifiable and other protected information into, say, ChatGPT, agencies lose control over how that data can be used. 

Additionally, by opening a shadow AI account at work, employees could agree to privacy and security terms they shouldn’t, Glasscock said. “If you’re an individual person signing up for something and you’re clicking yes [to terms and conditions], maybe those terms aren’t actually sufficient for what your state requires,” she said. “When you say yes with your employee email, you’re agreeing to those terms for your state, not just yourself.”

Although agencies might not be able to eliminate shadow AI, they can take actions to reduce it. Creating a policy around AI use is critical, Glasscock said.

“It’s very likely that people are using it … so employees probably should be aware of how it can best be used,” she said. “Policies should lay out what should this should be used for, what should it not be used for, what are the data leak risks or security risks associated with it.” 

To formulate a policy, governments have plenty of guidance. In December, NASCIO issued a 12-step AI blueprint for agencies starting to incorporate AI into their operations to inventory and document existing AI applications and create acquisition and development guidelines. Government leaders can also look to the National Institute of Standards and Technology’s AI Risk Management Framework and the Trustworthy and Responsible AI Resource Center.

Some states have taken legal steps to regulate their own AI use. For example, beginning Feb. 1, Connecticut’s Department of Administrative Services must perform ongoing assessments of systems that use AI, and the Office of Policy and Management must establish rules around the development, procurement, implementation, use and assessment of AI systems.

During the 2023 legislative session, at least 25 states and Washington, D.C., introduced AI bills, and 18 adopted resolutions or enacted legislation, according to the National Conference of State Legislatures. Louisiana’s Joint Committee on Technology and Cybersecurity, for example, is studying AI’s impact on operations, procurement and policy, and Texas created an AI advisory council to study and monitor AI systems developed, used or procured by state agencies. North Dakota and West Virginia have similar groups.

“Last year seemed like the year of generative AI. Everybody was like, ‘What is this new thing? We’re trying to get our head around it,’” Glasscock said. “My hope is that 2024 will be more the year of AI governance, putting in place those policies, road maps and useful legislation around it. We had a year to figure it out, and now it’s time to get organized.”

X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.