The time is now: Harnessing rigorous evaluation in a post-ARPA America

ogichobanov via Getty Images

 

Connecting state and local government leaders

COMMENTARY | Cities and states can use the federal funds to evaluate the effectiveness of new programs.

State and local governments are at the forefront of addressing historic levels of income inequality, rising housing instability and the impacts of climate change. They can supercharge their ability to create change by leading a movement of evidence building that will inform equitable programming and support economic mobility for generations.

The 2021 American Rescue Plan Act provided $350 billion to state and local governments to pilot new interventions to tackle the country’s most critical issues and collect evidence on how well their solutions perform. Evaluation––using research methods to examine the effectiveness of a program or policy in achieving a particular goal––gives agencies actionable information on how programs can most effectively support the community members they serve. 

Governments have until the end of this year to decide how remaining ARPA funds will be spent (and until 2026 to spend the funds). Now is the time for local governments to use this funding to set up evaluations of key programs and policies made possible by ARPA.

Leaders can make the most of this opportunity by thoughtfully identifying the questions in need of answers for decision-making, connecting with researchers to develop and implement rigorous evaluation, and sharing findings broadly to support other jurisdictions in advancing effective and equitable evidence-based approaches.

Identifying policy and implementation questions in need of evidence

King County in Washington has ambitious goals to advance equity and reduce climate impact. Existing data shows that fossil fuel use in residents’ homes is one of the region’s biggest contributors to climate change. To address both climate and equity goals, the county wants to ensure all households can convert aging fossil-fuel home heating to more effective and cleaner alternatives. King County currently offers a program that provides fully subsidized heat pump installations to low-income homeowners. New heat pumps not only replace oil or aging heating systems, which leads to increased energy efficiency in the home, but they also provide home cooling in a warming region where households with lower incomes tend to lack air conditioning. 

King County officials want to effectively and equitably reach homeowners with low incomes, but there is little existing evidence to guide how to best find and approach those households. A rigorous evaluation will help identify which recruitment methods (for example, in-person workshops or an interactive online application) are most effective in helping different homeowner populations complete an application for a new heat pump. Through this evaluation, the county expects to better target its outreach to ensure it best meets the needs of residents who will most benefit from the program. 

Connecting with researchers to rigorously evaluate innovations

In order to conduct this evaluation work, King County is partnering with The Abdul Latif Jameel Poverty Action Lab (J-PAL) affiliated researchers. This research partnership has had immediate pay-off as analysis of existing data has quickly led to a better understanding of who does and does not take up the existing program. The evaluation will build further reliable knowledge instrumental in shaping the county’s continued investments to meet equity and climate goals, long after ARPA funding runs out. County officials plan to share lessons learned so that other communities may benefit as well. 

King County’s work to develop evidence on clean energy use is part of a broader movement of jurisdictions investing in similar rigorous learning. Many other governments are also using ARPA funding to build and share evidence. Both the State of Connecticut and the City of Baltimore have budgeted funds to coordinate evaluation work and evidence use. Maine has also budgeted for its Department of Labor to review the outcomes of state workforce initiatives. These efforts will position Connecticut, Baltimore and Maine to make evidence-informed decisions about spending for years to come and support other state and local governments to do the same using the new evidence-base they have built. 

State and local leaders looking for help maximizing their impact by building evaluation capacity have a variety of partners to choose from. Some governments have internal evaluation teams that can help, like Minnesota Management and Budget’s Impact Evaluation Unit or San Diego County’s Office of Evaluation, Performance, and Analytics. In other places, local university centers, like California Policy Lab, support several jurisdictions. Private evaluation firms, like MDRC or Mathematica, help build evidence alongside government partners. Finally, J-PAL North America and Results for America’s LEVER program provide free training, workshops and technical assistance to build governments’ internal evaluation capacity.

When government leaders embed evaluation within their work to foster equity and economic mobility, they’re investing in building knowledge on how to best support their communities. This work is not easy. States and localities should make the most of the historic opportunity to devote ARPA funding—as well as future federal funding—to evaluation. This investment will help them sustain their programs and ensure they are best positioned to support the communities they serve for the long term.

Carrie S. Cihak is the evidence and impact officer for King County, Washington. Louise Geraghty is the government partnerships lead at The Abdul Latif Jameel Poverty Action Lab (J-PAL) North America. Zachary Markovits is the vice president and local practice lead at Results for America.

X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.